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(57) There are included: an initial sub-signal crea- 
tion section (1) which creates, from an original signal, 
sub-signals of shorter length than the original signal; a 
created sub-signal selection section (2) which, for the 
sub-signals produced by the initial sub-signal creation 
section (1 ), prunes the created sub-signal candidates to 
those for which the amount of data is less than for the 
original signal; a sub-signal re-creation section (3) which 
determines a created sub-signal which is actually to be 
used, using the created sub-signai candidates produced 
by the created sub-signal selection section (2); a com- 
pression mapping determination section (4) which de- 
termines, from the sub-signals produced by the sub-sig- 
nal re-creation section (3), a mapping for calculation of 
a compressed signal; and a signal compression section 
(5) which calculates a compressed signal correspond- 
ing to the sub-signals obtained by the sub-signal re-cre- 
ation section (3), based upon the mapping obtained by 
the compression mapping determination section (4). 
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Description 

BACKGROUND OF THE INVENTION 
5 Field of the Invention 

[0001] The present invention relates to a signal compression method which is suitable for representing an enormous 
signal sequence with a small amount of information, to a device, to a program and to a recording medium. The present 
invention also relates to a signal retrieval method which, along with utilizing the signal compression method, is suitable 
io for finding out the position of a signal which is similar to a signal which has been registered in advance from among 
an enormous signal sequence, to a device, to a program, and to a recording medium. 

Description of Related Art 

15 [0002] In the past, as signal retrieval methods for finding a position of a signal within a colossal signal sequence 
which is similar to a signal which has been inputted, there have been proposed several per se known high speed signal 
retrieval methods which find at high speed a position in a similar signal; for example, Japanese Patent 3,065,314 
(Patent Reference #1), and Japanese Unexamined Patent Application, First Publication No. 2001-092486 (Patent 
Reference #2). 

20 [0003] Furthermore, with regard to the above described type of signal retrieval method, as a method which reduces 
the cost of calculation for matching features one at a time by performing feature compression of the signal, there have 
been proposed methods for reducing the dimensions of the features in an effective manner (for example, Non-Patent 
Reference #3, specified below) by piecewise linear mapping which takes advantage of the continuity of a time series 
signal. These methods create mapping by segmenting a signal evenly into equal segments regardless of the properties 

25 of the signal. Furthermore, it is per se known to vary the length of the segments according to the properties of the 
signal (for example, see Non-Patent Reference #1: E. Keogh, K. Chakrabarti, S. Mehrotra, and M. Pazzani, "Locally 
adaptive dimensionality reduction for indexing large time series databases", Proc. of ACM SIGMOD Conference, pp. 
151-162, 2001; or Non-Patent Reference #2: C. Wang and S. Wang, "Supporting content-based searches on time 
series via approximation", Proc. of International Conference on Scientific and Statistical Database Management 

30 (SSDBM2000), pp. '69-81 , 2000). 

[0004] Furthermore, in the above type of signal retrieval method, as a method for locally eliminating matching pro- 
cedures which are not required, there is a per se known time series active search method in which the lower limit value 
of the distance is calculated in advance of matching by taking advantage of the properties of the histograms which are 
the features utilized for matching, and matching procedures are eliminated insofar as that a threshold which has been 

35 decided upon in advance is not exceeded; for example, see Patent Reference #1 : Japanese Patent 3,065,314, and 
Patent Reference #2: Japanese Unexamined Patent Application, First Publication No. 2001-092486. 
[0005] Furthermore, as a technique for signal compression which is utilized in the above kind of signal retrieval 
method, there is a per se known signal compression method which can retain feature sequences with a small memory 
capacity and can perform decisions relating to similarity between signals at high speed by reducing the dimensions of 
feature sequences which have been extracted from an audio signal. This signal compression technique is utilized in 
the above described signal retrieval method for finding out the position from among a signal sequence of a signal which 
is similar to another signal which has been registered in advance, such as when detecting the instant during a broadcast 
audio signal at which a specified tune has been played, or the like, and it is capable of being executed at high speed, 
in connection with the concrete signal compression methods which are used for such a technique, there are per se 

45 known signal compression methods for segmenting and compressing an original signal which has been provided in 
advance; for example, see Non-Patent Reference #3: "A Quick Search Method for Video Signals Based on Piecewise 
Linear Maps" by Akisato Kimura and three others, Technical Report of IEICE, The Institute of Electronics, Information 
and Communication Engineers, February 2002, Vol. 101 , No. 653, pp. 75-80. 

[0006] Here, with the high speed signal retrieval methods of the above described Patent Reference #1 and Patent 
50 Reference #2, for an absolutely colossal signal sequence, there may be a problem that it is not possible to find a similar 
signal within a sufficiently short time period, and there is a problem that it is not possible to deal with features other 
than histograms. 

[0007] Furthermore, with a signal compression method such as described in the above Non-Patent Reference #1 or 
Non-Patent Reference #2, it has become possible to perform signal compression at higher compression ratios than 
55 heretofore, there is the problem that a colossal amount of processing is required for determining the most suitable 
method of signal division. 

[0008] Yet further, with a signal compression method such as described in the above Non-Patent Reference #3, 
although the sub-signal may be built up by segmenting the original signal evenly and equally regardless of the properties 
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of that signal, alternatively, it is also possible to vary the length of the segments according to the properties of the 
signal. Accordingly, it has become possible to perform signal compression at better compression ratios than heretofore, 
with this method, there is the problem that a colossal amount of processing has become necessary for determining 
the function for performing compression. 

SUMMARY OF THE INVENTION 

[0009] The present invention has been conceived in the light of the above described problems, and its objective is, 
while avoiding a colossal amount of preprocessing labor, and along with performing the step of signal compression 
with a signal compression method which is better adapted to the properties of the object signal than conventional signal 
compression methods, to provide a signal compression method, device, program therefor, and recording medium, and 
a signal retrieval method, device, program therefor, and recording medium which utilize the signal compression method, 
which are able to represent signal sequences with a smaller amount of information, with which the efficiency of calcu- 
lation is higher than in the conventional techniques, which ensure uniform retrieval results, with which the amount of 
temporary storage which is required for the retrieval is not enormously increased, and with which it is possible to 
perform signal retrieval at higher speed. 

[0010] The present invention has been conceived in order to solve the above described problems, and the present 
invention is a signal compression method for compressing an original signal which has been provided in advance to 
convert the original signal into a compressed signal, comprising: an initial sub-signal creation step of creating, from 
the original signal, sub-signals of shorter length than the original signal; a created sub-signal selection step of, for each 
of the sub-signals which have been produced by the initial sub-signal creation step, pruning the created sub-signal 
candidates to those for which the amount of data is less than for the original signal; a sub-signal re-creation step of 
determining upon a created sub-signal which is actually to be used, using the created sub-signal candidates which 
have been produced by the created sub-signal selection step; a compression mapping determination step of deter- 
mining, from the respective sub-signals which have been produced by the sub-signal re-creation step, a mapping for 
calculation of a compressed signal; and a signal compression step of calculating a compressed signal which corre- 
sponds to each of the sub-signals which have been obtained by the sub-signal re-creation step, based upon the map- 
ping which has been obtained by the compression mapping determination step. 

[001 1] In addition, in the present invention, the signal compression step comprises: a signal mapping step of mapping 
each of the sub-signals which have been obtained by the sub-signal re-creation step by the mapping which has been 
obtained by the compression mapping determination step; a projection distance calculation step of calculating, for the 
sub-signal after the mapping which has been obtained by the signal mapping step, the distance from the sub-signal 
which has been obtained by the sub-signal re-creation step; and a compressed feature creation step of creating a 
compressed signal, based upon the respective sub-signals after mapping which have been produced by the signal 
mapping step and the projection distance which has been produced by the projection distance calculation step. 
[0012] In addition, in the present invention, the initial sub-signal creation step segments the original signal from the 
top of the original signal, and takes the sub-signal after the segmentation as its resulting sub-signal. 
[001 3] In addition, in the present invention, the created sub-signal selection step and the sub-signal re-creation step 
determine segmentation boundaries in order from the top of the original signal. 

[0014] in addition, in the present invention, the created sub-signal selection step and the sub-signal re-creation step 
set a segmentation boundary shiftable width which is determined in advance, and, taking the segmentation boundary 
which has been obtained by the initial sub-signal creation step as a reference, determine segmentation boundaries 
which are to be actually utilized within a segmentation boundary shiftable range having the segmentation boundary 
shiftable width on both sides of the center thereof. 

[001 5] In addition, in the present invention, the created sub-signai selection step shifts the segmentation boundaries 
to some locations and calculates compression ratios, and, based upon the results thereof, selects a range in which 
the segmentation boundaries which are to be actually utilized can exist. 

[0016] In addition, in the present invention, the created sub-signal selection step automatically determines the 
number of times for calculation of compression ratio in the created sub-signal selection step, so as to reduce the number 
of times of calculation of compression ratio in the created sub-signal selection step and the sub-signal re-creation step. 
[0017] In addition, in the present invention, the initial sub-signal creation step extracts features from the original 
signal, and uses the extracted features, represented as a sequence of multi-dimensional vectors, as a new original 
signal. 

[0018] In addition, the present invention is a signal retrieval method for, at any location within a stored signal, which 
Is an original signal which is registered in advance, calculating the distance from a reference signal, which is a signal 
which is taken as an object, and finding a location from the stored signal which is similar to the reference signal, 
comprising: the steps which are comprised in the signal compression method as described above; a reference feature 
extraction step in which a feature is produced from the reference signal; a stored feature extraction step in which a 
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window upon which attention is focused is set within the stored signal, and in which a feature is produced from the 
stored signal within the window upon which attention is focused; a reference feature compression step in which a 
reference feature which has been produced by the reference feature extraction step is compressed, based upon the 
mapping which has been produced by the compression mapping determination step; a feature matching step in which 

5 the distance is calculated between a reference compressed signal which has been produced by the reference feature 
compression step and a stored compressed signal which has been produced from the signal compression step by 
newly using the feature sequence which has been produced by repeatedly performing the processing of the stored 
feature extraction step while shifting the window upon which attention is focused; and a signal detection decision step 
in which, by comparing together the distance which has been produced by the feature matching step and a search 

10 threshold, which is a threshold which corresponds to the distance, it is decided whether or not the reference signal is 
present at the location within the stored signal, wherein the processing of the feature matching step and the processing 
of the signal detection decision step are repeated while shifting the window upon which attention is focused. 
[001 9] In addition, in the above-described signal retrieval method, the present invention further comprising: a distance 
re-calculation step in which, for the location in the database signal at which it has been decided by the signal detection 

is decision step that the query signal is present, the distance between the feature sequence which has been produced 
by the reference feature extraction step and the feature sequence which has been produced by the stored feature 
extraction step is calculated; and a signal detection re-decision step in which, by comparing together the distance 
which has been produced by the distance re-calculation step and the search threshold, it is re-decided whether or not 
the query signal is present at the location of the database signal, wherein the processing of the feature matching step, 

20 the signal detection decision step, the distance re-calculation step, and the signal detection re-decision step is repeated 
while shifting the window upon which attention is focused; for some locations within the database signal, the distance 
from the query signal is calculated; and it is determined whether or not the query signal is present at the locations 
within the database signal. 

[0020] In addition, in the above-described signal retrieval method, the present invention, further comprising a skip 

25 width calculation step in which, based upon the distance which has been calculated by the feature matching step, a 
skip width for the window upon which attention is focused is calculated, and the window upon which attention is focused 
is shifted by the skip width, wherein the processing of the feature matching step, the signal detection decision step, 
and the skip width calculation step is repeated while shifting the window upon which attention is focused; for some 
locations within the database signal, the distance from the query signal is calculated; and it is determined whether or 

30 not the query signal is present at the locations within the database signal. 

[0021] In addition, the present invention is a signal compression device which compresses an original signal which 
is provided in advance to convert the original signal into a compressed signal, comprising: an initial sub-signal creation 
section which creates, from the original signal, sub-signals of shorter length than the original signal; a created sub- 
signal selection section which, for each of the sub-signals which has been produced by the initial sub-signal creation. 

35 section, prunes the created sub-signal candidates to those for which the amount of data is less than for the original 
signal; a sub-signal re-creation section which, using the created sub-signal candidates which have been produced by 
the created sub-signal selection section, determines upon a created sub-signal which is actually to be used; a com- 
pression mapping determination section which determines upon a mapping for calculation of a compressed signal from 
the respective sub-signals which has been obtained by the sub-signal re-creation section; and a signal compression 

40 section which calculates a compressed signal which corresponds to each of the sub-signals which have been obtained 
by the sub-signal re-creation section, based upon the mapping which has been obtained by the compression mapping 
determination section. 

[0022] In addition, the present invention is a signal retrieval device for, at any location within a stored signal, which 
is an original signal which is registered in advance, calculating the distance from a reference signal, which is a signal 
*3 which is taken as an object, and finding a location from the stored signal which is similar to the reference signal, 
comprising: the sections which are comprised in the signal compression device as described above; a reference feature 
extraction section which produces a feature from the reference signal; a stored feature extraction section which sets 
a window upon which attention is focused within the stored signal, and which produces a feature from the stored signal 
within the window upon which attention is focused; a reference feature compression section which compresses a 
so reference feature which has been produced by the reference feature extraction section, based upon the mapping which 
has been produced by the compression mapping determination section; a feature matching section which calculates 
the distance between a reference compressed signal which has been produced by the reference feature compression 
section and a stored compressed signal which has been produced from the signal compression section by newly using 
the feature sequence which has been produced by repeatedly performing the processing by the stored feature extrac- 
ts tion section while shifting the window upon which attention is focused; and a signal detection decision section which, 
by comparing together the distance which has been produced by the feature matching section and a search threshold, 
which is a threshold which corresponds to the distance, decides whether or not the reference signal is present at the 
location within the stored signal, wherein the operation of the feature matching section and the operation of the signal 
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detection decision section are repeated while shifting the window upon which attention is focused. 
[0023] In addition, in the above-described signal retrieval device, the present invention further comprising: a distance 
re-calculation section which, for the location in the database signal at which it has been decided by the signa! detection 
decision section that the query signal is present, calculates the distance between the feature sequence which has been 

s produced by the reference feature extraction section and the feature sequence which has been produced by the stored 
feature extraction section; and a signal detection re-decision section which, by comparing together the distance which 
has been produced by the distance re-calculation section and the search threshold, re-decides whether or not the 
query signal is present at the location of the database signal, wherein the processing of the feature matching section, 
the signal detection decision section, the distance re-calculation section, and the signal detection re-decision section 

10 is repeated while shifting the window upon which attention is focused; for some locations within the database signal, 
the distance from the query signal is calculated; and it is determined whether or not the query signal is present at the 
locations within the database signal. 

[0024] In addition, in the above-described signal retrieval device, the present invention further comprising a skip 
width calculation section which, based upon the distance which has been calculated by the feature matching section, 

is calculates a skip width for the window upon which attention is focused, and shifts the window upon which attention is 
focused by the skip width, and wherein the processing of the feature matching section, the signal detection decision 
section, and the skip width calculation section is repeated while shifting the window upon which attention is focused; 
for some locations within the database signal, the distance from the query signal is calculated; and it is determined 
whether or not the query signal is present at the locations within the database signal. 

20 [0025] In addition, the present invention is a signal compression program which compresses an original signal which 
has been provided in advance to convert the original signal into a compressed signal, for causing a computerto execute: 
an initial sub-signal creation step of creating, from the original signal, sub-signals of shorter length than the original 
signal; a created sub-signal selection step of, for each of the sub-signals which have been produced by the initial sub- 
signal creation step, pruning the created sub-signal candidates to those for which the amount of data is less than for 

25 the original signal; a sub-signal re-creation step of determining upon a created sub-signal which is actually to be used, 
using the created sub-signal candidates which have been produced by the created sub-signal selection step; a com- 
pression mapping. determination step of determining, from each of the sub-signals which have been produced by the 
sub-signal re-creation step, a mapping for calculation of a compressed signal; and a signal compression step of cal- 
culating a compressed signal which corresponds to each of the sub-signals which have been obtained by the sub- 

30 signal re-creation step, based upon the mapping which has been obtained by the compression mapping determination 
step. 

[0026] In addition, the present invention is a signal retrieval program for, at any location within a stored signal, which 
is an original signal which is registered in advance, calculating the distance from a reference signal, which is a signal 
which is taken as an object, and finding a location from the stored signal which is similar to the reference signal, for 

35 causing a computerto execute: the steps which are comprised in the signal compression program as described above; 
a reference feature extraction step in which a feature is produced from the reference signal; a stored feature extraction 
step in which a window upon which attention is focused is set within the stored signal, and in which a feature is produced 
from the stored signal within the window upon which attention is focused; a reference feature compression step in 
which a reference feature which has been produced by the reference feature extraction step is compressed, based 

*o upon the mapping which has been produced by the compression mapping determination step; a feature matching step 
in which the distance is calculated between a reference compressed signal which has been produced by the reference 
feature compression step and a stored compressed signal which has been produced from the signal compression step 
by newly using the feature sequence which has been produced by repeatedly performing the processing of the stored 
feature extraction step while shifting the window upon which attention is focused; a signal detection decision step in 

^5 which, by comparing together the distance which has been produced by the feature matching step and a search thresh- 
old, which is a threshold which corresponds to the distance, it is decided whether or not the reference signal is present 
at the location within the stored signal; and a step of repeatedly executing the feature matching step and the signal 
detection decision step while shifting the window upon which attention is focused. 

[0027] In addition, in the above-described signal retrieval program, the present invention further comprising: a dis-. 

50 tance re-calculation step in which, for the location in the database signal at which it has been decided by the signal 
detection decision step that the query signal is present, the distance between the feature sequence which has been 
produced by the reference feature extraction step and the feature sequence which has been produced by the stored 
feature extraction step is calculated; and a signal detection re-decision step in which, by comparing together the dis- 
tance which has been produced by the distance re-calculation step and the search threshold, it is re-decided whether 

55 or not the query signal is present at the location of the database signal, and wherein a step is performed in which the 
processing of the feature matching step, the signal detection decision step, the distance re-calculation step, and the 
signal detection re-decision step is repeated while shifting the window upon which attention is focused, for some lo- 
cations within the database signal, the distance from the query signal is calculated; and it is determined whether or 
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not the query signal is present at the locations within the database signal. 

[0028] in addition, in the above-described signal retrieval program, the present invention further comprising a skip 
width calculation step in which, based upon the distance which has been calculated by the feature matching step, a 
skip width for the window upon which attention is focused is calculated, and the window upon which attention is focused 

5 is shifted by the skip width, and wherein a step is performed in which the processing of the feature matching step, the 
signal detection decision step, and the skip width calculation step is repeated while shifting the window upon which 
attention is focused, for some locations within the database signal, the distance from the query signal is calculated; 
and it is determined whether or not the query signal is present at the locations within the database signal. 
[0029] In addition, the present invention is a recording medium capable of being read by a computer, upon which is 

10 recorded a signal compression program which compresses an original signal which has been provided in advance to 
convert the original signal into a compressed signal, for causing a computer to execute: an initial sub-signal creation 
step of creating, from the original signal, sub-signals of shorter length than the original signal; a created sub-signal 
selection step of, for each of the sub-signals which have been produced by the initial sub-signal creation step, pruning 
the created sub-signal candidates to those for which the amount of data is less than for the original signal; a sub-signal 

is re-creation step of determining upon a created sub-signal which is actually to be used, using the created sub-signal 
candidates which have been produced by the created sub-signal selection step; a compression mapping determination 
step of determining, from each of the sub-signals which have been produced by the sub-signal re-creation step, a 
mapping for calculation of a compressed signal; and a signal compression step of calculating a compressed signal 
which corresponds to each of the sub-signals which have been obtained by the sub-signal re-creation step, based 

20 upon the mapping which has been obtained by the compression mapping determination step. 

[0030] In addition, the present invention is a recording medium capable of being read by a computer, upon which is 
recorded a signal retrieval program for, at any location within a stored signal, which is an original signal which is 
registered in advance, calculating the distance from a reference signal, which is a signal which is taken as an object, 
and finding a location from the stored signal which is similar to the reference signal, for causing a computer to execute: 

25 the steps which are comprised in the signal compression program as described above; a reference feature extraction 
step in which a feature is produced from the reference signal; a stored feature extraction step in which a window upon 
which attention is focused is set within the stored signal, and in which a feature is produced from the stored signal 
within the window upon which attention is focused; a reference feature compression step in which a reference feature 
which has been produced by the reference feature extraction step is compressed, based upon a mapping which has 

30 been produced by the compression mapping determination step; a feature matching step in which the distance is 
calculated between a reference compressed signal which has been produced by the reference feature compression 
step and a stored compressed signal which has been produced from the signal compression step by newly using the 
feature sequence which has been produced by repeatedly performing the processing of the stored feature extraction 
step while shifting the window upon which attention is focused; a signal detection decision step in which, by comparing 

35 together the distance which has been produced by the feature matching step and a search threshold, which is a thresh- 
old which corresponds to the distance, it is decided whether or not the reference signal is present at the location within 
the stored signal; and a step of repeatedly executing the feature matching step and the signal detection decision step 
while shifting the window upon which attention is focused. 

[0031] In addition, in the recording medium according to the present invention, the signal retrieval program further 

40 comprising: a distance re-calculation step in which, for the location in the database signal at which it has been decided 
by the signal detection decision step that the query signal is present, the distance between the feature sequence which 
has been produced by the reference feature extraction step and the feature sequence which has been produced by 
the stored feature extraction step is calculated; and a signal detection re-decision step in which, by comparing together 
the distance which has been produced by the distance re-calculation step and the search threshold, it is re-decided 

45 whether or not the query signal is present at the location of the database signal, and wherein a step is performed in 
which the processing of the feature matching step, the signal detection decision step, the distance re-calculation step, 
and the signal detection re-decision step is repeated while shifting the window upon which attention is focused, for 
some locations within the database signal, the distance from the query signal is calculated; and it is determined whether 
or not the query signal is present at the locations within the database signal. 

so [0032] In addition, in the recording medium according to the present invention, the signal retrieval program further 
comprising a skip width calculation step in which, based upon the distance which has been calculated by the feature 
matching step, a skip width for the window upon which attention is focused is calculated, and the window upon which 
attention is focused is shifted by the skip width, and wherein a step is performed in which the processing of the feature 
matching step, the signal detection decision step, and the skip width calculation step is repeated while shifting the 

55 window upon which attention is focused, for some locations within the database signal, the distance from the query 
signal is calculated, and it is determined whether or not the query signal is present at the locations within the database 
signal. 

[0033] According to the present invention, by utilizing a novel signal compression method or a novel signal compres- 
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si on device which comprises a sub-signal creation step and a sub-signal re-creation step, and by adaptively varying 
the length of the sub-signals according to the properties of the signal while avoiding a colossal amount of pre-processing 
labor, thus compressing the original signal which has been provided in advance further than in the conventional tech- 
niques, it is possible to represent the signal sequences with a smaller amount of information than heretofore. 

5 [0034] Furthermore, by applying this signal compression method or signal compression device to a signal retrieval 
method or a signal retrieval device which finds out, from a stored signal which has been registered in advance, portions 
which are similar to a reference signal which constitutes an objective, it is possible to perform information compression 
of the feature information, and accordingly it is also possible to obtain the beneficial results that, along with being able 
to anticipate great increase in the speed of retrieval, it is also possible to reduce the amount of stored information. 

10 [0035] In addition, the present invention is a signal retrieval method which finds out portions from a database signal 
which has been registered in advance which are similar to a query signal which is taken as an object, comprising: a 
query feature extraction step in which a feature is produced from the query signal; a database feature extraction step 
in which a window upon which attention is focused is set within the database signal, and in which a feature is produced 
from the database signal within the window upon which attention is focused; a database feature partitioning step in 

15 which a feature sequence which has been produced by repeatedly performing the database feature extraction step 
while shifting the window upon which attention is focused is partitioned; a database feature pruning step in which a 
representative feature is extracted from the feature sequence which has been obtained after partitioning by the data- 
base feature partitioning step, and a representative feature sequence is produced which consists of a smaller number 
of features; a feature region extraction step in which a region is produced in which a feature which is included in the 

20 partition which has been produced by the database feature partitioning step is present; a feature matching step in 
which a distance is calculated between a feature sequence which has been produced by the query feature extraction 
step and a representative feature sequence which has been produced by the database feature pruning step; a distance 
compensation step in which the distance which has been calculated by the feature matching step is compensated 
using the region which has been produced by the feature region extraction step; and a signal detection decision step 

25 in which, by comparing together the distance which has been produced after compensation by the distance compen- 
sation step and a search threshold, which is a threshold which corresponds to the distance, it is decided whether or 
not the query signal is present at the location within the database signal, and wherein the processing of the feature 
matching step through the signal detection decision step is repeated while shifting the window upon which attention is 
focused, for some locations within the database signal, the distance from the query signal is calculated, and it is de- 

30 termined whether or not the query signal is present at the locations within the database signal. 

[0036] According to the present invention, in comparison with conventional signal retrieval methods, it is possible to 
guarantee identical retrieval results, and moreover it is possible to perform the retrieval at a higher speed by locally 
eliminating matches which are not required, with almost no increase in the memory capacity which is required for the 
retrieval step. 

35 [0037] In addition, in the above-described signal retrieval method according to the present invention, in the database 
feature pruning step, any single feature in the partition is taken as a representative feature. 

[0038] In addition, in the above-described signal retrieval method according to the present invention, in the database 
feature pruning step, the centroid of the features in the partition is taken as a representative feature. 
[0039] In addition, in the above-described signal retrieval method according to the present invention, in the database 
-♦o feature partitioning step, the feature sequence which has been produced by repeatedly performing the processing of 
the database feature extraction step while shifting the window upon which attention is focused is segmented equally 
into lengths which have been specified in advance. 

[0040] In addition, in the above-described signal retrieval method according to the present invention, in the database 
feature partitioning step, the feature sequence which has been produced by repeatedly performing the processing of 
45 the database feature extraction step while shifting the window upon which attention is focused is segmented so that 
the region in which a feature is present which is produced by the feature region extraction step becomes smaller than 
a maximum region which has been specified in advance. 

[0041] In addition, in the above-described signal retrieval method, the present invention further comprising: a seg- . 
ment extraction step in which segments, which are sub-sequences, are extracted by segmenting a feature sequence 

so which has been produced by repeatedly performing the database feature extraction step while shifting the window 
upon which attention is focused; a compression mapping determination step in which, from each of the segments which 
have been obtained by the segment extraction step, a mapping is determined for calculation of a feature of less di- 
mensions than the feature; a database feature compression step in which a feature which corresponds to a segment 
which has been obtained by the segment extraction step and which is of less dimensions than the feature is calculated 

55 based upon the mapping which has been obtained by the compression mapping determination step; and a query 
feature compression step in which a feature which corresponds to a feature which has been obtained by the query 
feature extraction step and which is of less dimensions than the feature is calculated based upon the mapping which 
has been obtained by the compression mapping determination step, and wherein, in the database feature pruning 
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step, a representative feature sequence is produced by taking the compressed feature sequence which has been 
produced by the database feature compression step as a new feature sequence, and, in the feature matching step, a 
matching is performed of the compressed feature which has been produced by the query feature compression step as 
a new feature, the processing of the feature matching step through the signal detection decision step is repeated while 
s shifting the window upon which attention is focused, for some locations within the database signal, the distance from 
the query signal is calculated; and it is determined whether or not the query signal is present at the locations within 
the database signal. 

[0042] According to the present invention, by greatly reducing the number of features to which the indices are as- 
signed, it is possible to perform the retrieval with a reduced amount of storage. 

10 [0043] In addition, in the above-described signal retrieval method according to the present invention, the database 
feature compression step comprises: a database feature mapping step in which a segment which has been obtained 
by the segment extraction step is mapped according to the mapping which has been obtained by the compression 
mapping determination step; a database projection distance calculation step in which, for the compressed feature 
sequence which has been produced by the database feature mapping step, the distance from the feature sequence 

is which has been produced by the database feature extraction step is calculated; and a database compressed feature 
creation step in which a new compressed feature sequence is created from the compressed feature sequence which 
has been produced by the database feature mapping step and the projection distance which has been produced by 
the database projection distance calculation step, and wherein the query feature compression step comprises: a query 
feature mapping step in which the feature which has been obtained by the query feature extraction step is mapped 

20 according to the mapping which has been obtained by the compression mapping determination step; a query projection 
distance calculation step in which* for the compressed feature which has been produced by the query feature mapping 
step, the distance from the feature which has been produced by the query feature extraction step is calculated; and a 
query compression feature creation step in which a new compressed feature is created from the compressed feature 
which has been produced by the query feature mapping step and the projection distance which has been produced by 

25 the query projection distance calculation step. 

[0044] In addition, in the above-described signal retrieval method according to the present invention, the compression 
mapping determination step extracts a representative feature by a Karhunen-Loeve transform. 

[0045] In addition, in the above-described signal compression method, the present invention further comprising: a 
distance re-calculation step in which, for the location in the database signal at which it has been decided by the signal 

30 detection decision step that the query signal is present, the distance between the feature which has been produced 
by the query feature extraction step and the feature sequence which has been produced by the database feature 
extraction step is calculated; and a signal detection re-decision step in which, by comparing together the distance 
which has been produced by the distance re-calculation step and the search threshold, it is again decided whether or 
not the query signal is present at the location of the database signal, and wherein the processing of the feature matching 

35 step, the signal compensation step, the signal detection decision step, the distance re-calculation step, and the signal 
detection re-decision step is repeated while shifting the window upon which attention is focused, for some locations 
within the database signal, the distance from the query signal is calculated; and it is determined whether or not the 
query signal is present at the locations within the database signal. 

[0046] In addition, in the above-described signal retrieval method, the present invention further comprising: a data- 
40 base feature classification step in which the respective features which have been produced by repeatedly performing 
the database feature extraction step while shifting the window upon which attention is focused are classified based 
upon a distance which has been defined in advance, and a representative feature of the classification is determined 
upon; a selection threshold setting step in which a selection threshold for the distance which has been defined by the 
database feature classification step is calculated from a search threshold which has been defined in advance; and a 
« database feature selection step in which, among the classification which has been produced by the database feature 
classification step, a feature is selected which is included in the classification which contains a representative feature 
such that the distance from the feature which has been produced by the query feature extraction step satisfies a 
condition which is produced from the selection threshold which has been calculated by the selection threshold setting 
step. 

so [0047] According to the present invention, it becomes possible further to prune the mappings which are used upon 
the signal which it is desired to search for, by reducing the extent of the range which it is required to search. Due to 
this, it is possible to reduce the cost of calculation. 

[0048] In addition, in the above-described signal retrieval method according to the present invention, the database 
feature classification step classifies the features based upon a vector quantization algorithm, using Euclid distance as 
55 a distance measure. 

[0049] In addition, in the above-described signal retrieval method according to the present invention, the feature 
matching step calculates the distance based upon Manhattan distance or Euclid distance. 

[0050] In addition, in the above-described signal retrieval method according to the present invention, the database 
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proj ection distance calculation step calculates the distance based upon Manhattan distance or Euclid distance. 
[0051] In addition, in the above-described signal retrieval method according to the present invention, the distance 
re-calculation step calculates the distance based upon Manhattan distance or Euclid distance. 
[0052] In addition, in the above-described signal retrieval method according to the present invention, the query feature 
5 extraction step and the database feature extraction step classify the features by a method which is determined fn 
advance, create a histogram which is a frequency distribution table for each classification, and output the histogram 
as a new feature. 

[0053] In addition, in the above-described signal retrieval method, the present invention further comprising a skip 
width calculation step in which, based upon the distance which has been calculated by the distance compensation 

10 step, a skip width for the window upon which attention is focused is calculated, and the window upon which attention 
is focused is shifted by the skip width, and wherein the processing of the feature matching step, the distance compen- 
sation step, the signal detection decision step, and the skip width calculation step is repeated while shifting the window 
upon which attention is focused, for some locations within the database signal, the distance from the query signal is 
calculated, and it is determined whether or not the query signal is present at the locations within the database signal. 

15 [0054] In addition, the present invention is a signal retrieval method which finds out portions from a database signal 
which has been registered in advance which are similar to a query signal which is taken as an object, comprising: a 
query feature extraction step in which a feature is produced from a query signal; a database feature extraction step in 
which a window upon which attention is focused is set within the database signal, and in which a feature is produced 
from the database signal within the window upon which attention is focused; a database feature classification step in 

20 which the respective features which have been produced by repeatedly performing the database feature extraction 
step while shifting the window upon which attention is focused are classified based upon a distance which has been 
defined in advance, and a representative feature of the classification is determined upon; a selection threshold setting 
step in which a selection threshold for the distance which has been defined by the database feature classification step 
is calculated from a search threshold which has been defined in advance; a database feature selection step in which, 

25 among the classification which has been produced by the database feature classification step, a feature is selected 
which is included in the classification which contains a representative feature such that the distance from the feature 
which has been produced by the query feature extraction step satisfies a condition which is produced from the selection 
threshold which has been calculated by the selection threshold setting step; a segment extraction step in which seg- 
ments, which are sub-sequences, are extracted by segmenting a feature sequence which has been produced by re- 

30 peatedly performing the database feature extraction step while shifting the window upon which attention is focused; a 
compression mapping determination step in which, from each of the segments which have been obtained by the seg- 
ment extraction step, a mapping is determined for calculation of a feature of less dimensions than the feature; a data- 
base feature compression step in which a feature which corresponds to a segment which has been obtained by the 
segment extraction step and which is of less dimensions than the feature is calculated based upon the mapping which 

35 has been obtained by the compression mapping determination step; a query feature compression step in which a 
feature which corresponds to a feature which has been obtained by the query feature extraction step and which is of 
less dimensions than the feature is calculated based upon the mapping which has been obtained by the compression 
mapping determination step; a feature matching step in which a distance is calculated between a compressed feature 
sequence which has been produced by the database feature compression step and a compressed feature which has 

40 been produced by the query feature extraction step; and a signal detection decision step in which, by comparing to- 
gether the distance which has been calculated by the feature matching step and a search threshold, which is a threshold 
which corresponds to the distance, it is decided whether or not the query signal is present at the location within the 
database signal, and wherein the processing of the feature matching step and the signal detection decision step is 
repeated while shifting the window upon which attention is focused, for some locations within the database signal, the 

45 distance from the query signal is calculated; and it is determined whether or not the query signal is present at the 
locations within the database signal 

[0055] According to the present invention, by reducing the range over which the search must be performed, it be- 
comes possible further to prune the mappings which are used upon the signal which it is desired to search for. Due to . 
this, it is possible to reduce the cost of calculation. 

so [0056] In addition, in the above-described signal retrieval method, the present invention further comprising: a distance 
re-calculation step in which, for the location in the database signal at which it has been decided by the signal detection 
decision step that the query signal is present, the distance between the feature which has been produced by the query 
feature extraction step and the feature sequence which has been produced by the database feature extraction step is 
calculated; and a signal detection re-decision step in which, by comparing together the distance which has been pro- 

55 duced by the distance re-calculation step and the search threshold, it is again decided whether or not the query signal 
is present at the location of the database signal, and wherein the processing of the feature matching step, the signal 
detection decision step, the distance re-caiculation step, and the signal detection re-decision step is repeated while 
shifting the window upon which attention is focused, for some locations within the database signal, the distance from 



9 



EP 1 426 899 



Page 10 of 103 



EP 1 426 899 A2 

the query signal is calculated; and it is determined whether or not the query signal is present at the locations within 
the database signal 

[0057] In addition, in the above-described signal retrieval method, the present invention further comprising a skip 
width calculation step in which, based upon the distance which has been calculated by the feature matching step, a 

s skip width for the window upon which attention is focused is calculated, and the window upon which attention is focused 
is shifted by the skip width, and wherein the processing of the feature matching step, the signal detection decision 
step, and the skip width calculation step is repeated while shifting the window upon which attention is focused, for 
some locations within the database signal, the distance from the query signal is calculated; and it is determined whether 
or not the query signal is present at the locations within the database signal. 

10 [0058] In addition, the present invention is a signal retrieval device which finds out portions from a database signal 
which has been registered in advance which are similar to a query signal which is taken as an object, comprising: a 
query feature extraction section which produces a feature from the query signal; a database feature extraction section 
which sets a window upon which attention is focused is set within the database signal, and which produces a feature 
from the database signal within the window upon which attention is focused; a database feature partitioning section 

is which partitions a feature sequence which has been produced by repeatedly performing the processing of the database 
feature extraction section while shifting the window upon which attention is focused; a database feature pruning section 
which extracts a representative feature from the feature sequence which has been obtained after partitioning by the 
database feature partitioning section, and which produces a representative feature sequence which consists of a small- 
er number of features; a feature region extraction section which produces a region in which a feature which is included 

20 in the partition which has been produced by the database feature partitioning section is present; a feature matching 
section which calculates a distance between a feature sequence which has been produced by the query feature ex- 
traction section and a representative feature sequence which has been produced by the database feature pruning 
section; a distance compensation section in which the distance which has been calculated by the feature matching 
section is compensated using the region which has been produced by the feature region extraction section; and a 

25 signal detection decision section in which, by comparing together the distance which has been produced after com- 
pensation by the distance compensation section and a search threshold, which is a threshold which corresponds to 
the distance, it is decided whether or not the query signal is present at the location within the database signal, and 
wherein the processing of the feature matching section through the signal detection decision section is repeated while 
shifting the window upon which attention is focused; for some locations within the database signal, the distance from 

30 the query signal is calculated; and it is determined whether or not the query signal is present at the locations within 
the database signal. 

[0059] In addition, in the above-described signal retrieval device, the present invention further comprising: a segment 
extraction section which extracts segments, which are sub-sequences, by segmenting a feature sequence which has 
been produced by repeatedly performing the processing of the database feature extraction section while shifting the 

35 window upon which attention is focused; a compression mapping determination section which, from each of the seg- 
ments which have been obtained by the segment extraction section, determines a mapping for calculation of a feature 
of less dimensions than the feature; a database feature compression section which calculates a feature which corre- 
sponds to a segment which has been obtained by the segment extraction section and which is of less dimensions than 
the feature based upon a mapping which has been obtained by the compression mapping determination section; and 

40 a query feature compression section which calculates a feature which corresponds to a feature which has been obtained 
by the query feature extraction section and which is of less dimensions than the feature based upon a mapping which 
has been obtained by the compression mapping determination section, and wherein, by the database feature pruning 
section, a representative feature sequence is produced by using a compressed feature sequence which has been 
produced by the database feature compression section is produced as a new feature sequence; by the feature matching 

^5 section, matching is performed using a compressed feature which has been produced by the query feature compression 
section as a new feature; and further: the processing of the feature matching section through the signal detection 
decision section is repeated while shifting the window upon which attention is focused, for some locations within the 
database signal, the distance from the query signal is calculated; and it is determined whether or not the query signal 
is present at the locations within the database signal. 

so [0060] In addition, in the above-described signal retrieval device, the present invention further comprising: a distance 
re-calculation section which, for the location in the database signal at which it has been decided by the signal detection 
decision section that the query signal is present, calculates the distance between the feature sequence which has been 
produced by the query feature extraction section and the feature sequence which has been produced by the database 
feature extraction section; and a signal detection re-decision section which, by comparing together the distance which 

55 has been produced by the distance re-calculation section and the search threshold, again decides whether or not the 
query signal is present at the location of the database signal, and wherein the processing of the feature matching 
section, the distance compensation section, the signal detection decision section, the distance re-calculation section, 
and the signal detection re-decision section is repeated while shifting the window upon which attention is focused, for 
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some locations within the database signal, the distance from the query signal is calculated; and it is determined whether 
or not the query signal is present at the locations within the database signal. 

[0061] In addition, in the above-described signal retrieval device, the present invention further comprising: a database 
feature classification section which classifies the respective features which have been produced by repeatedly per- 
forming the processing of the database feature extraction section while shifting the window upon which attention is 
focused based upon a distance which has been determined in advance, and determines upon a representative feature 
of the classification; a selection threshold setting section which calculates a selection threshold for the distance which 
has been defined by the database feature classification section from a search threshold which has been defined in 
advance; and a database feature selection section which, for the classification which has been produced by the data- 
base feature classification section, selects a feature which is included in the classification which contains a represent- 
ative feature such that the distance from the feature which has been produced by the query feature extraction section 
satisfies a condition which is produced from the selection threshold which has been calculated by the selection threshold 
setting section. 

[0062] In addition, in the above-described signal retrieval device, the present invention further comprising a skip 
width calculation section which, based upon the distance which has been calculated by the distance compensation 
section, calculates a skip width for the window upon which attention is focused, and shifts the window upon which 
attention is focused by the skip width, and wherein the processing of the feature matching section, the distance com- 
pensation section, the signal detection decision section, and the skip width calculation section is repeated while shifting 
the window upon which attention is focused, for some locations within the database signal, the distance from the query 
signal is calculated; and it is determined whether or not the query signal is present at the locations within the database 
signal. 

[0063] In addition, the present invention is a signal retrieval device which finds out portions from a database signal 
which has been registered in advance which are similar to a query signal which is taken as an object, comprising: a 
query feature extraction section which produces a feature from a query signal; a database feature extraction section 
which sets a window upon which attention is focused within the database signal, and which produces a feature from 
the database signal within the window upon which attention is focused; a database feature classification section which 
classifies the respective features which have been produced by repeatedly performing the processing of the database 
feature extraction section while shifting the window upon which attention is focused based upon a distance which has 
been determined in advance, and determines upon a representative feature of the classification; a selection threshold 
setting section which calculates a selection threshold for the distance which has been defined by the database feature 
classification section from a search threshold which has been defined in advance; a database feature selection section 
which, among the classification which has been produced by the database feature classification section, selects a 
feature which is included in the classification which contains a representative feature such that the distance from the 
feature which has been produced by the query feature extraction section satisfies a condition which is produced from 
the selection threshold which has been calculated by the selection threshold setting section; a segment extraction 
section which extracts segments, which are sub-sequences, by segmenting a feature sequence which has been pro- 
duced by repeatedly performing the processing of the database feature extraction section while shifting the window 
upon which attention is focused; a compression mapping determination section which, from each of the segments 
which have been obtained by the segment extraction section, determines a mapping for calculation of a feature of less 
dimensions than the feature; a database feature compression section which calculates a feature which corresponds 
to a segment which has been obtained by the segment extraction section and which is of less dimensions than the 
feature based upon a mapping which has been obtained by the compression mapping determination section; a query 
feature compression section which calculates a feature which corresponds to a feature which has been obtained by 
the query feature extraction section and which is of less dimensions than the feature based upon the mapping which 
has been obtained by the compression mapping determination section; a feature matching section which calculates a 
distance between a compressed feature sequence which has been produced by the database feature compression 
section and a compressed feature which has been produced by the query feature extraction section; and a signal 
detection decision section which, by comparing together the distance which has been calculated by the feature match- 
ing section and a search threshold, which is a threshold which corresponds to the distance, decides whether or not 
the query signal is present at the location within the database signal, and wherein the processing of the feature matching 
section through the signal detection decision section is repeated while shifting the window upon which attention is 
focused, for some locations within the database signal, the distance from the query signal is calculated; and it is de- 
termined whether or not the query signal is present at the locations within the database signal. 
[0064] In addition, in the above-described signal retrieval device, the present invention further comprising: a distance 
re-calculation section which, for the location in the database signal at which it has been decided by the signal detection 
decision section that the query signal is present, calculates the distance between the feature sequence which has been 
produced by the query feature extraction section and the feature sequence which has been produced by the database 
feature extraction section; and a signal detection re-decision section which, by comparing together the distance which 
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has been produced by the distance re-calculation section and the search threshold, again decides whether or not the 
query signal is present at the location of the database signal and wherein the processing of the feature matching 
section, the signal detection decision section, the distance re-calculation section, and the signal detection re-decision 
section is repeated while shifting the window upon which attention is focused, for some locations within the database 
5 signal, the distance from the query signal is calculated; and it is determined whether or not the query signal is present 
at the locations within the database signal. 

[0065] In addition, in the above-described signal retrieval device, the present invention further comprising a skip 
width calculation section which, based upon the distance which has been calculated by the feature matching section, 
calculates a skip width for the window upon which attention is focused, and shifts the window upon which attention is 
10 focused by the skip width and wherein the processing of the feature matching section, the signal detection decision 
section, and the skip width calculation section is repeated while shifting the window upon which attention is focused; 
for some locations within the database signal, the distance from the query signal is calculated; and it is determined 
whether or not the query signal is present at the locations within the database signal. 

[0066] In addition, the present invention is a program for execution by a computer of a signal retrieval device which 

15 finds out portions from a database signal which has been registered in advance which are similar to a query signal 
which is taken as an object, comprising: a query feature extraction step in which a feature is produced from the query 
signal; a database feature extraction step in which a window upon which attention is focused is set within the database 
signal, and in which a feature is produced from the database signal within the window upon which attention is focused; 
a database feature partitioning step in which a feature sequence which has been produced by repeatedly performing 

20 the database feature extraction step while shifting the window upon which attention is focused is partitioned; a database 
feature pruning step in which a representative feature is extracted from the feature sequence which has been obtained 
after partitioning by the database feature partitioning step, and a representative feature sequence is produced which 
consists of a smaller number of features; a feature region extraction step in which a region is produced in which a 
feature which is included in the partition which has been produced by the database feature partitioning step is present; 

25 a feature matching step in which a distance is calculated between a feature which has been produced by the query 
feature extraction step and a representative feature sequence which has been produced by the database feature 
pruning step; a distance compensation step in which the distance which has been calculated by the feature matching 
step is compensated using the region which has been produced by the feature region extraction step; a signal detection 
decision step in which, by comparing together the distance which has been produced after compensation by the dis- 

30 tance compensation step and a search threshold, which is a threshold which corresponds to the distance, it is decided 
whether or not the query signal is present at the location within the database signal; and a step of: repeating the 
processing of the feature matching step through the signal detection decision step whiie shifting the window upon which 
attention is focused, for some locations within the database signal, calculating the distance from the query signal; and 
determining whether or not the query signal is present at the locations within the database signal. 

35 [0067] In addition, in the above-described program, the present invention further comprising: a segment extraction 
step in which segments, which are sub-sequences, are extracted by segmenting a feature sequence which has been 
produced by repeatedly performing the database feature extraction step while shifting the window upon which attention 
is focused; a compression mapping deterfni nation step in which, from each of the segments which have been obtained 
by the segment extraction step, a mapping is determined for calculation of a feature of less dimensions than the feature; 

40 a database feature compression step in which a feature which corresponds to a segment which has been obtained by 
the segment extraction step and which is of less dimensions than the feature is calculated based upon the mapping 
which has been obtained by the compression mapping determination step; a query feature compression step in which 
a feature which corresponds to a feature which has been obtained by the query feature extraction step and which is 
of less dimensions than the feature is calculated based upon the mapping which has been obtained by the compression 

^5 mapping determination step; and a step in which, in the database feature pruning step, a representative feature se- 
quence is produced by taking the compressed feature sequence which has been produced by the database feature 
compression step as a new feature sequence, and, in the feature matching step, a matching is performed of the com- 
pressed feature which has been produced by the query feature compression step as a new feature; the processing of 
the feature matching step through the signal detection decision step is repeated while shifting the window upon which 

so attention is focused; for some locations within the database signal, the distance from the query signal is calculated; 
and it is determined whether or not the query signal is present at the locations within the database signal. 
[0068] In addition, in the above-described program, the present invention further comprising: a distance re-calculation 
step in which, for the location in the database signal at which it has been decided by the signal detection decision step 
that the query signal is present, the distance between the feature which has been produced by the query feature 

55 extraction step and the feature sequence which has been produced by the database feature extraction step is calcu- 
lated; a signal detection re-decision step in which, by comparing together the distance which has been produced by 
the distance re-calculation step and the search threshold, it is again decided whether or not the query signal is present 
at the location of the database signal; and a step in which: the processing of the feature matching step, the distance 
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compensation step, the signal detection decision step, the distance re-calculation step, and the signal detection re- 
decision step is repeated while shifting the window upon which attention is focused; for some locations within the 
database signal, the distance from the query signal is calculated; and it is determined whether or not the query signal 
is present at the locations within the database signal. 

s [0069] in addition, in the above-described program, the present invention further comprising: a database feature 
classification step in which the respective features which have been produced by repeatedly performing the database 
feature extraction step while shifting the window upon which attention is focused are classified based upon a distance 
which has been determined in advance, and a representative feature of the classification is determined upon; a selection 
threshold setting step in which a selection threshold for the distance which has been defined by the database feature 

10 classification step is calculated from a search threshold which has been defined in advance; and a database feature 
selection step in which, among the classification which has been produced by the database feature classification step, 
a feature is selected which is included in the classification which contains a representative feature such that the distance 
from the feature which has been produced by the query feature extraction step satisfies a condition which is produced 
from the selection threshold which has been calculated by the selection threshold setting step. 

15 [0070] In addition, in the above-described program, the present invention further comprising: a skip width calculation 
step in which, based upon the distance which has been calculated by the distance compensation step, a skip width 
for the window upon which attention is focused is calculated, and the window upon which attention is focused is shifted 
by the skip width; and a step in which: the processing of the feature matching step, the distance compensation step, 
the signal detection decision step, and the skip width calculation step is repeated while shifting the window upon which 

20 attention is focused; for some locations within the database signal, the distance from the query signal is calculated; 
and it is determined whether or not the query signal is present at the locations within the database signal. 
[0071] In addition, the present invention is a program for execution by a computer of a signal retrieval device which 
finds out portions from a database signal which has been registered in advance which are similar to a query signal 
which is taken as an object, comprising: a query feature extraction step in which a feature is produced from the query 

25 signal; a database feature extraction step in which a window upon which attention is focused is set within the database 
signal, and in which a feature is produced from the database signal within the window upon which attention is focused; 
a database feature classification step in which the respective features which have been produced by repeatedly per- 
forming the database feature extraction step while shifting the window upon which attention is focused are classified 
based upon a distance which has been determined in advance, and a representative feature of the classification is 

30 determined upon; a selection threshold setting step in which a selection threshold for the distance which has been 
defined by the database feature classification step is calculated from a search threshold which has been defined in 
advance; a database feature selection step in which, among the classification which has been produced by the database 
feature classification step, a feature is selected which is included in the classification which contains a representative 
feature such that the distance from the feature which has been produced by the query feature extraction step satisfies 

35 a condition which is produced from the selection threshold which has been calculated by the selection threshold setting 
step; a segment extraction step in which segments, which are sub-sequences, are extracted by segmenting a feature 
sequence which has been produced by repeatedly performing the database feature extraction step while shifting the 
window upon which attention is focused; a compression mapping determination step in which, from each of the seg- 
ments which have been obtained by the segment extraction step, a mapping is determined for calculation of a feature 

40 of less dimensions than the feature; a database feature compression step in which a feature which corresponds to a 
segment which has been obtained by the segment extraction step and which is of less dimensions than the feature is 
calculated based upon the mapping which has been obtained by the compression mapping determination step; a query 
feature compression step in which a feature which corresponds to a feature which has been obtained by the query 
feature extraction step and which is of less dimensions than the feature is calculated based upon the mapping which 

45 has been obtained by the compression mapping determination step; a feature matching step in which a distance is 
calculated between a compressed feature sequence which has been produced by the database feature compression 
step and a compressed feature which has been produced by the query feature extraction step; a signal detection 
decision step in which, by comparing together the distance which has been calculated by the feature matching step . 
and a search threshold, which is a threshold which corresponds to the distance, it is decided whether or not the query 

50 signal is present at the location within the database signal; and a step in which: the processing of the feature matching 
step and the signal detection decision step is repeated while shifting the window upon which attention is focused; for 
some locations within the database signal, the distance from the query signal is calculated; and it is determined whether 
or not the query signal is present at the locations within the database signal. 

[0072] In addition, in the above-described program, the present invention further comprising: a distance re-calculation 
55 step in which, for the location in the database signal at which it has been decided by the signal detection decision step 
that the query signal is present, the distance between the feature which has been produced by the query feature 
extraction step and the feature sequence which has been produced by the database feature extraction step is calcu- 
lated; a signal detection re-decision step in which, by comparing together the distance which has been produced by 
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the distance re-calculation step and the search threshold, it is again decided whether or not the query signal is present 
at the location of the database signal; and a step in which: the processing of the feature matching step, the signal 
detection decision step, the distance re-calculation step, and the signal detection re-decision step is repeated while 
shifting the window upon which attention is focused; for some locations within the database signal, the distance from 
5 the query signal is calculated; and it is determined whether or not the query signal is present at the locations within 
the database signal. 

[0073] In addition, in the above-described program, the present invention further comprising: a skip width calculation 
step in which, based upon the distance which has been calculated by the feature matching step, a skip width for the 
window upon which attention is focused is calculated, and the window upon which attention is focused is shifted by 
10 the skip width; and a step in which: the processing of the feature matching step, the signal detection decision step, 
and the skip width calculation step is repeated while shifting the window upon which attention is focused; for some 
locations within the database signal, the distance from the query signal is calculated; and it is determined whether or 
not the query signal is present at the locations within the database signal. 

[0074] In addition, the present invention is a recording medium capable of being read by a computer, upon which is 

is recorded a program for causing a computer of a signal retrieval device which finds out portions from a database signal 
which has been registered in advance which are similar to a query signal which is taken as an object to execute: a 
query feature extraction step in which a feature is produced from the query signal; a database feature extraction step 
in which a window upon which attention is focused is set within the database signal, and in which a feature is produced 
from the database signal within the window upon which attention is focused; a database feature partitioning step in 

20 which a feature sequence which has been produced by repeatedly performing the database feature extraction step 
while shifting the window upon which attention is focused is partitioned; a database feature pruning step in which a 
representative feature is extracted from the feature sequence which has been obtained after partitioning by the data- 
base feature partitioning step, and a representative feature sequence is produced which consists of a smaller number 
of features; a feature region extraction step in which a region is produced in which a feature which is included in the 

25 partition which has been produced by the database feature partitioning step is present; a feature matching step in 
which a distance is calculated between a feature which has been produced by the query feature extraction step and 
a representative feature sequence which has been produced by the database feature pruning step; a distance com- 
pensation step in which the distance which has been calculated by the feature matching step is compensated using 
the region which has been produced by the feature region extraction step; a signal detection decision step in which, 

30 by comparing together the distance which has been produced after compensation by the distance compensation step 
and a search threshold, which is a threshold which corresponds to the distance, it is decided whether or not the query 
signal is present at the location within the database signal; and a step of: repeating the processing of the feature 
matching step through the signal detection decision step while shifting the window upon which attention is focused; 
for some locations within the database signal, calculating the distance from the query signal; and determining whether 

35 or not the query signal is present at the locations within the database signal. 

[0075] In addition, in the above-described recording medium according to the present invention, the above-described 
program further comprising: a segment extraction step in which segments, which are sub-sequences, are extracted 
by segmenting a feature sequence which has been produced by repeatedly performing the database feature extraction 
step while shifting the window upon which attention is focused; a compression mapping determination step in which, 

40 from each of the segments which haves been obtained by the segment extraction step, a mapping is determined for 
calculation of a feature of less dimensions than the feature; a database feature compression step in which a feature 
which corresponds to a segment which has been obtained by the segment extraction step and which is of less dimen- 
sions than the feature is calculated based upon the mapping which has been obtained by the compression mapping 
determination step; a query feature compression step in which a feature which corresponds to a feature which has 

45 been obtained by the query feature extraction step and which is of less dimensions than the feature is calculated based . 
upon the mapping which has been obtained by the compression mapping determination step; and a step in which, in 
the database feature pruning step, a representative feature sequence is produced by taking the compressed feature 
sequence which has been produced by the database feature compression step as a new feature sequence, and, in 
the feature matching step, a matching is performed of the compressed feature which has been produced by the query 

50 feature compression step as a new feature; the processing of the feature matching step through the signal detection 
decision step is repeated while shifting the window upon which attention is focused; for some locations within the 
database signal, the distance from the query signal is calculated; and it is determined whether or not the query signal 
is present at the locations within the database signal. 

[0076] In addition, in the above-described recording medium according to the present invention, the above-described 
55 program further comprising: a distance re-calculation step in which, for the location in the database signal at which it 
has been decided by the signal detection decision step that the query signal is present, the distance between the 
feature which has been produced by the query feature extraction step and the feature sequence which has been 
produced by the database feature extraction step is calculated; a signal detection re-decision step in which, by com- 
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paring together the distance which has been produced by the distance re-calculation step and the search threshold, 
It is again decided whether or not the query signal is present at the location of the database signal; and a step in which: 
the processing of the feature matching step, the distance compensation step, the signal detection decision step, the 
distance re-calculation step, and the signal detection re-decision step is repeated while shifting the window upon which 

5 attention is focused; for some locations within the database signal, the distance from the query signal is calculated; 
and it is determined whether or not the query signal is present at the locations within the database signal. 
[0077] In addition, in the above-described recording medium according to the present invention, the above-described 
program further comprising: a database feature classification step in which the respective features which have been 
produced by repeatedly performing the database feature extraction step while shifting the window upon which attention 

10 is focused are classified based upon a distance which has been determined in advance, and a representative feature 
of the classification is determined upon; a selection threshold setting step in which a selection threshold for the distance 
which has been defined by the database feature classification step is calculated from a search threshold which has 
been defined in advance; and a database feature selection step in which, among the classification which has been 
produced by the database feature classification step, a feature is selected which is included in the classification which 

15 contains a representative feature such that the distance from the feature which has been produced by the query feature 
extraction step satisfies a condition which is produced from the selection threshold which has been calculated by the 
selection threshold setting step. 

[0078] In addition, in the above-described recording medium according to the present invention, the above-described 
program further comprising: a skip width calculation step in which, based upon the distance which has been calculated 

20 by the distance compensation step, a skip width for the window upon which attention is focused is calculated, and the 
window upon which attention is focused is shifted by the skip width; and a step in which: the processing of the feature 
matching step, the distance compensation step, the signal detection decision step, and the skip width calculation step 
is repeated while shifting the window upon which attention is focused; for some locations within the database signal, 
the distance from the query signal is calculated; and it is determined whether or not the query signal is present at the 

25 locations within the database signal. 

[0079] In addition, the present invention is a recording medium capable of being read by a computer, upon which is 
recorded a program for causing a computer of a signal retrieval device which finds out portions from a database signal 
which has been registered in advance which are similar to a query signal which is taken as an object to execute: a 
query feature extraction step in which a feature is produced from the query signal; a database feature extraction step 

30 in which a window upon which attention is focused is set within the database signal, and in which a feature is produced 
from the database signal within the window upon which attention is focused; a database feature classification step in 
which the respective features which have been produced by repeatedly performing the database feature extraction 
step while shifting the window upon which attention is focused are classified based upon a distance which has been 
determined in advance, and a representative feature of the classification is determined upon; a selection threshold 

35 setting step in which a selection threshold forthe distance which has been defined by the database feature classification 
step is calculated from a search threshold which has been defined in advance; a database feature selection step in 
which, among the classification which has been produced by the database feature classification step, a feature is 
selected which is included in the classification which contains a representative feature such that the distance from the 
feature which has been produced by the query feature extraction step satisfies a condition which is produced from the 

40 selection threshold which has been calculated by the selection threshold setting step; a segment extraction step in 
which segments, which are sub-sequences, are extracted by segmenting a feature sequence which has been produced 
by repeatedly performing the database feature extraction step while shifting the window upon which attention is focused; 
a compression mapping determination step in which, from each of the segments which have been obtained by the 
segment extraction step, a mapping is determined for calculation of a feature of less dimensions than the feature; a 

45 database feature compression step in which a feature which corresponds to the segment which has been obtained by 
the segment extraction step and which is of less dimensions than the feature is calculated based upon the mapping 
which has been obtained by the compression mapping determination step; a query feature compression step in which 
a feature which corresponds to a feature which has been obtained by the query feature extraction step and which is . 
of less dimensions than the feature is calculated based upon the mapping which has been obtained by the compression 

50 mapping determination step; a feature matching step in which a distance is calculated between a compressed feature 
sequence which has been produced by the database feature compression step and a compressed feature which has 
been produced by the query feature extraction step; a signal detection decision step in which, by comparing together 
the distance which has been calculated by the feature matching step and a search threshold, which is a threshold 
which corresponds to the distance, it is decided whether or not the query signal is present at the location within the 

55 database signal; and a step in which: the processing of the feature matching step and the signal detection decision 
step is repeated while shifting the window upon which attention is focused; for some locations within the database 
signal, the distance from the query signal is calculated; and it is determined whether or not the query signal is present 
at the locations within the database signal. 
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[0080] In addition, in the above-described recording medium according to the present invention, the above-described 
program further comprising: a distance re-calculation step in which, for the location in the database signal at which it 
has been decided by the signal detection decision step that the query signal is present, the distance between the 
feature which has been produced by the query feature extraction step and the feature sequence which has been 
produced by the database feature extraction step is calculated; a signal detection re-decision step in which, by com- 
paring together the distance which has been produced by the distance re-calculation step and the search threshold, 
it is again decided whether or not the query signal is present at the location of the database signal; and a step in which: 
the processing of the feature matching step, the signal detection decision step, the distance re-calculation step, and 
the signal detection re-decision step is repeated while shifting the window upon which attention is focused; for some 
locations within the database signal, the distance from the query signal is calculated; and it is determined whether or 
not the query signal is present at the locations within the database signal. 

[0081] In addition, in the above-described recording medium according to the present invention, the above-described 
program further comprising: a skip width calculation step in which, based upon the distance which has been calculated 
by the feature matching step, a skip width for the window upon which attention is focused is calculated, and the window 
upon which attention is focused is shifted by the skip width; and a step in which: the processing of the feature matching 
step, the signal detection decision step, and the skip width calculation step is repeated while shifting the window upon 
which attention is focused; for some locations within the database signal, the distance from the query signal is calcu- 
lated; and it is determined whether or not the query signal is present at the locations within the database signal. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0082] 

FIG. 1 is a block diagram showing the structure of a first preferred embodiment of the present invention. 
FIG. 2 is a block diagram showing the structure of a second preferred embodiment of the present invention. 
FIG. 3 is a block diagram showing the structure of a third preferred embodiment of the present invention. 
FIG. 4 is a block diagram showing the structure of a fourth preferred embodiment of the present invention. 
FIG. 5 is a flow chart showing the processing operation of the signal compression device according to the first 
preferred embodiment of the present invention shown in FIG. 1. 

FIG. 6 is a flow chart showing the operation of an initial sub-signal creation section 1 shown in FIG. 1 . 
FIG. 7 is a flow chart showing the operation of a created sub-signal selection section 2 which is shown in FIG. 1 . 
FIG. 8 is a flow chart showing the overall flow of an average dimensionality calculation procedure shown in FIG. 7. 
FIG. 9 is a flow chart showing the flow of a base extraction procedure which is used in the average dimensionality 
calculation procedure shown in FIG. 8. 

FIG. 10 is a flow chart showing the operation of a sub-signal re-creation section 3 shown in FIG. 1 . 

FIG. 11 is a flow chart showing the operation of a compression mapping determination section shown in FIG. 1 . 

FIG. 1 2 is a flow chart showing the overall operation of a signal compression section 5 shown in FIG. 1 . 

FIG. 13 is a flow chart showing the steps of a signal mapping procedure shown in FIG. 12. 

FIG. 14 is a flow chart showing the steps of a projection distance calculation procedure shown in FIG. 12. 

FIG. 15 is a flow chart showing the steps of a compression feature creation procedure shown in FIG. 12. 

FIG. 1 6 is a flow chart showing the processing operation of signal retrieval devices according to the second, the 

third, and the fourth preferred embodiments of the present invention. 

FIG. 1 7 is a flow chart showing the operation of a reference feature extraction section 6 shown in FIG. 2. 

FIG. 1 8 is a flow chart showing the operation of a stored feature extraction section 7 shown in FIG. 2. 

FIG. 19 is a flow chart showing the operation of a reference feature compression section 8 shown in FIG. 2. 

FIG. 20 is a flow chart showing the steps of a reference signal mapping procedure shown in FIG. 19. 

FIG. 21 is a flow chart showing the steps of a reference projection distance calculation procedure shown in FIG. 1 9. 

FIG. 22 is a flow chart showing the steps of a reference compression feature creation procedure shown in FIG. 19. 

FIG. 23 is a flow chart showing the operation of a feature matching section 9 shown in FIG. 2. 

FIG. 24 is a flow chart showing the operation of a signal detection decision section 1 0 shown in FIG. 2. 

FIG. 25 is a flow chart showing the operation of a skip width calculation section 11 shown in FIG. 3. 

FIG. 26 is a flow chart showing the operation of a distance re-calculation section 12 shown in FIG. 4. 

FIG. 27 is a flow chart showing the operation of a signal detection re-decision section 1 shown in FIG. 4. 

FIG. 28 is a first explanatory figure showing the results of certain experiments.. 

FIG. 29 is a second explanatory figure showing the results of certain experiments. 

FIG. 30 is a figure showing a functional block of a signal retrieval device to which the method according to the fifth 
preferred embodiment of the present invention has been applied. 

FIG. 31 is a figure showing a functional block of a signal retrieval device to which the signal retrieval method 
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according to the sixth preferred embodiment of the present invention has been applied. 

FIG. 32 is a figure showing a functional block of a signal retrieval device to which the signal retrieval method 
according to the seventh preferred embodiment of the present invention has been applied. 
FIG. 33 is a figure showing a functional block of a signal retrieval device to which the signal retrieval method 
5 according to the eighth preferred embodiment of the present invention has been applied. 

FIG. 34 is a figure showing a functional block of a signal retrieval device to which the signal retrieval method 
according to the ninth preferred embodiment of the present invention has been applied. 

FIG. 35 is a figure showing a functional block of a signal retrieval device to which the signal retrieval method 

according to the tenth preferred embodiment of the present invention has been applied. 
10 FIG. "36 is a figure showing a functional block of a signal retrieval device to which the signal retrieval method 

according to the eleventh preferred embodiment of the present invention has been applied. 

FIG. 37 is a figure showing a functional block of a signal retrieval device to which the signal retrieval method 

according to the twelfth preferred embodiment of the present invention has been applied. 

FIG. 38 is a flow chart showing the processing which is performed by a database feature partitioning section. 
is FIG. 39 is a flow chart showing the processing which is performed by a database feature pruning section. 

FIG. 40 is a flow chart showing the processing which is performed by a feature region extraction section. 

FIG. 41 is a second flow chart showing the processing which is performed by a feature matching section. 

FIG. 42 is a flow chart showing the processing which is performed by a distance compensation section. 

FIG. 43 is a flow chart showing the processing which is performed by a segment extraction section. 
20 FIG. 44 is a flow chart showing the processing which is performed by a database feature classification section. 

FIG. 45 is a flow chart showing the processing which is performed by a selection threshold setting section. 

FIG. 46 is a figure showing the processing which is performed by a database feature selection section. 

FIG. 47 is a figure showing the way in which a histogram space is defined. 

FIG. 48 is a third explanatory figure showing the results of certain experiments. 
25 FIG. 49 is a fourth explanatory figure showing the results of certain experiments. 

FIG. 50 is a fifth explanatory figure showing the results of certain experiments. 

FIG. 51 is a sixth explanatory figure showing the results of certain experiments. 

FIG. 52 is a figure showing a functional block of a signal retrieval device to which the method according to the 
thirteenth preferred embodiment of the present invention has been applied. 
30 FIG, 53 is a figure showing a functional block of a signal retrieval device to which the method according to the 

fourteenth preferred embodiment of the present invention has been applied. 

FIG. 54 is a figure showing a functional block of a signal retrieval device to which the method according to the 
fifteenth preferred embodiment of the present invention has been applied. 

FIG. 55 is a third flow chart showing the processing which is performed by a feature matching section. 
35 FIG. 56 is a seventh explanatory figure showing the results of certain experiments. 

FIG. 57 is an eighth explanatory figure showing the results of certain experiments. 

DETAILED DESCRIPTION OF THE INVENTION 

40 [0083] In the following signal compression devices according to various preferred embodiments of the present in- 
vention will be described with reference to the appended drawings. While the present invention can be applied to 
various types of signals which are to be the objects of processing, as the example which will be employed in this 
specification, along with the object signal which is to be subjected to the processing (the original signal) being built up 
from a video signal, it will be supposed that it employs a histogram sequence which is an example of a mutti-dimensional 

45 vector. Note that a histogram is a frequency distribution table which has been obtained for each of certain classifications 
by, for example, extracting certain features from the video signal, and by classifying these features with a method which 
has been set in advance. Furthermore, if it is possible to extract the features of the original signal as a multi-dimensional 
vector sequence, it will be acceptable for the original signal to utilize this mufti-dimensional vector sequence, without 
limitation to a histogram. 

50 

- THE FIRST PREFERRED EMBODIMENT - 

[0084] FIG. 1 is a block diagram showing the structure of the first preferred embodiment of the present invention. In 
this figure, the reference symbol 1 denotes an initial sub-signal creation section which creates, from the original signal, 
55 various sub-signals whose lengths are shorter than that of the original signal. The reference symbol 2 denotes a created 
sub-signal selection section which, for each sub-signal which is outputted from the initial sub-signal creation section 
1 , reduces the candidates for creation of sub-signals to those for which the amount of data is smaller than that of the 
original signal. The reference symbol 3 denotes a sub-signal re-creation section which, by using the candidates for 
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created sub-signal which have been outputted by the created sub-signal selection section 2, determines which created 
sub-signal should actually be used. The reference symbol 4 denotes a compression mapping determination section 
which, from each sub-signal which has been outputted from the sub-signal re-creation section 3, determines upon a 
mapping for calculation of a compressed signal. The reference symbol 5 denotes a signal compression section which 
5 calculates the compressed signals which correspond to each sub-signal which has been outputted from the sub-signal 
re-creation section 3, based upon the mappings which have been outputted from the compression mapping determi- 
nation section 4. 

[0085] Note that this signal compression section 5 comprises: a signal mapping section 51 which, using the sub- 
signals which have been outputted from the sub-signal re-creation section 3 and the set of linear mappings which have 

10 been outputted from the compression mapping determination section 4, projects the histogram within each of the sub- 
signals into a subspace which has been created from its sub-signal; a projection distance calculation section 52 which, 
using the sub-signals which are outputted from the sub-signal re-creation section 3, the sets of linear mappings which 
are outputted from the compression mapping determination section 4, and the sets of compression histogram sequence 
which are outputted from the signal mapping section 51, calculates the distance between each histogram and the 

15 compression histogram which corresponds thereto; and a compression feature creation section 53 which, using the 
set of compression histogram sequence which has been outputted from the signal mapping section 51 and the pro- 
jection distances which have been outputted from the projection distance calculation section 52, calculates the se- 
quence of compression features. 

[0086] The signal compression device which is shown in FIG. 1 is one which takes as its input a histogram sequence 
20 which has been extracted from the original signal, in other words from a video signal which it is desired to compress, 
and which outputs a compressed signal, in other words a compression histogram sequence resulting from compression 
of the histogram sequence extracted from the video signal. 

[0087] Next, the operation of the signal compression device which is shown in FIG. 1 will be explained while referring 
to FIGS. 5 through 15. First, the overall operational flow of the signal compression device according to the first preferred 
25 embodiment of the present invention will be explained while referring to the flow chart which shows the processing 
operation of the signal compression device shown in FIG. 5. The detailed of each procedure which the overall flow 
includes will be explained hereinafter. 

[0088] In FIG. 5, first, the initial sub-signal creation section 1 reads in the original signal which has been supplied (in 
a step S1). Next, the initial sub-signal creation section 1 performs an initial sub-signal creation procedure (in a step 

30 S2). Next, the created sub-signal selection section 2 performs a created sub-signal selection procedure (in a step S3). 
Next, the sub-signal re-creation section 3 performs a sub-signal re-creation procedure (in a step S4). Next, the com- 
pression mapping determination section 4 performs a compression mapping determination procedure (in a step S5). 
Furthermore, the signal compression section 5 performs a signal compression procedure (in a step S6). The signal 
compression section 5 outputs the final compressed signal, which corresponds to the original signal (in a step S7). 

35 [0089] Next, the details of the initial sub-signal creation procedure (the step S2) which is shown in FIG. 5 will be 
explained while referring to FIG. 6. FIG. 6 is a flow chart showing the operation of the initial sub-signal creation section 1 . 
[0090] First, the initial sub-signal creation section 1 reads in the sequence of histograms which is the original signal 
(in a step S11). Next, the initial sub-signal creation section 1 segments the histogram sequence which has been read 
in into equal segments, according to a number of segments which has been provided in advance (in a step S1 2). The 

^o initial sub-signal creation section 1 outputs the set of segments resulting from this division, which is a histogram se- 
quence (in a step S 13). 

[0091] Next, the details of the created sub-signal selection procedure (the step S3) shown in FIG. 5 will be explained 
with reference to FIG. 7. FIG. 7 is a flow chart showing the operation of the created sub-signal selection section 2. 
[0092] In FIG. 7, first, the created sub-signal selection section 2 reads in the set of fixed length segments (sub- 

45 signals) which is outputted from the initial sub-signal creation section 1 (in a step S 1 5). Next, a segmentation boundary 
shiftable width, which is the width over which the segmentation boundary can be shifted from the current position, is 
set in advance, and, for each segmentation boundary, a range having the segmentation boundary shiftable width on 
both sides of the current position of the segmentation boundary is set as a segmentation boundary shiftable range (in 
a step S16). Next, among the segmentation boundaries of the segments, attention is paid (in a step S 17) to the first 

50 segmentation boundary. For two segments which have a segmentation boundary in common, the dimensionality of the 
compressed signal when the segmentation boundary is at the current position is calculated, and the average value 
with these normalized by segment length is calculated (in a step S 18). 

[0093] Next, for two segments which have a segmentation boundary in common, the dimensionality of the com- 
pressed signal which the segmentation boundary is at one of the both ends of the segmentation boundary shiftable 
55 range is calculated, and the average values with these normalized by segment length are calculated (in a step S 1 9). 
[0094] Furthermore, the average value of the dimensionality for several locations within the segmentation boundary 
shiftable range is calculated in the same manner. The number of times to perform this calculation is obtained from the 
average value of the dimensionality at the segmentation boundary at the above described three locations. Next, the 
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15 



average value of the dimensionality is calculated in order from the top of the segmentation boundary shiftabfe range 
so as to be spaced at equal intervals within the segmentation boundary shiftable range, based upon the number of 
times for calculation which has thus been obtained (in steps S20, S21 , and S22). 

[0095] It is then determined whether or not change in the dimensionality is present for any among the histogram 
sequence which have the segmentation boundary in common (in a step S23), and, when a change has occurred in 
the dimensionality for any among the histogram sequence which have the segmentation boundary in common (YES 
in the step S23), then all the positions in the range from the present calculation location to the calculation location one 
before are retained as candidates for the most suitable segmentation boundary (in a step S24). Note that the details 
of the method by which the average value of the dimensionality is calculated will be explained hereinafter with reference 
to the figures. 

[0096] Furthermore, the number x of locations for performing the calculation at the one side of the segmentation 
boundary shiftable range is obtained in the following manner. 

[0097] First, in the procedures which are performed by the created sub-signal selection section 2 and by the sub- 
signal re-creation section 3 which will be described hereinafter, the number of locations f(x) for calculation of the average 
value of the dimensionality which is necessary for determining a single segmentation boundary is given by the following. 



20 



/(*) = |(2x + 3) + r-^-jx2, 
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[0098] K is obtained as follows: 
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K = 



(C lc - C^) + mmtCfiCiCig) - minCC^ ,C^) 
if Clr > Crr* Cll < Crl> — Q?c 

if Clr > Crr-> Cll < Crl> ^lc > Crc 
0. Otherwise 
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where A is the width of the segmentation boundary shiftable range, C LL , C LC> and are the dimensionality of 
the segment of the top side, when the segmentation boundary is at respectively the front end, the initial position, and 
the rear end of the segmentation boundary shiftable range, and C RL , C RC , and C RR are the dimensionality of the 
segment of the end side, when the segmentation boundary is at respectively the front end, the initial position, and the 
rear end of the segmentation boundary shiftable range. 
[0099] This function f(x) takes its minimum value when 



so [0100] The closest integer to this x is set as the number of locations for performing the calculation. 

[0101] Since the number of times that the calculation is to be performed is obtained in this manner, if the calculation 
for all the calculation locations has not been completed (NO in the step S25), then the boundary is shifted to the next 
calculation location (in a step S26), and the operations in the steps S22 through S24 are repeated. 
[0102] Furthermore, if the calculation for all the calculation locations has been completed (YES in the step S25), 

S5 then a decision is made as to whether or not the operations have been completed for all of the segmentation boundaries 
(in a step S27), and, if the operations have in fact not been completed for all of the segmentation boundaries (NO in 
the step S27), then the segmentation boundary is changed to the next segmentation boundary (in a step S28), and 
the operations in the steps S1 8 through S26 are repeated. On the other hand, when the operations have in fact been 
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completed for all of the segmentation boundaries (YES in the step S27), then the created sub-signal selection section 
2 outputs the set of segmentation boundary candidates which have been retained (in a step S29). 
[0103] Next the method by which the average values of the dimensionality above described are calculated will be 
explained while referring to FIGS. 8 and 9. FIG. 8 is a flow chart showing the overall flow of the average dimensionality 
5 calculation procedure, while FIG. 9 is a flow chart showing the flow of a base extraction procedure which is used in 
this average dimensionality calculation procedure. In concrete terms, the average value of the dimensionality is cal- 
culated as described below. 

[0104] Referring to FIG. 8, first, two segments which have a segmentation boundary in common are read in (in a 
step S31). Next, from the two segments which have been presented, the base of a subspace which well represents 
10 the property of the original signal is extracted (in a step S32). 

[01 05] Note that, as will be described in the subsequent explanation of the extraction method for the base performed 
in the step S32 using FIG. 9, first, for each segment which has been presented (in a step S36), a KL (Karhunert-Loeve) 
transform is performed (in a step S3 7). In concrete terms, this KL transform is performed according to the steps de- 
scribed below. First, the average histogram of the histograms within the segment and the covariance matrix are cal- 
ls culated. For the j-th segment 
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(/= 1 ,2,— ,M) the covariance matrix SO) is calculated in the following manner: 

25 

1=1 

30 

where M is the number of segments, Lj is the length of the j-th segment, iW is the average histogram of X®, and 
(•) T is the transposed matrix. 

[0106] Next, the eigenvalues and the eigenvectors of the covariance matrix S(i) (for j=1 , 2, ... M) are obtained. The 
above are the steps for the KL transform. 

35 [0107] Note that the value which is obtained by dividing the eigenvalue which corresponds to each eigenvector which 
has been obtained by the KL transform by the total value of the eigenvalues of all the eigenvectors is termed the 
contribution for that eigenvector. Next, the eigenvectors are rearranged in order of size of contribution, and the eigen- 
vectors are selected in order until the total value of the contribution is greater than a contribution threshold which is 
given in advance (in steps S3 8 and S39), and, by taking the eigenvectors which have been selected as the bases of 

40 a subspace (in a step S40), a base set is obtained (in a step S41). 

[0108] In a step S33 of FIG. 8, since the number Nj of the bases (where j=1 , 2, ... M) which has been extracted from 
each segment by doing the above is the dimensionality of the compressed signal, thus, as in the following, the average 
values Nj" (where j=1 , 2, ... M-1) are calculated by normalizing them by the respective segment lengths (in the step S33). 

45 L L 

[0109] The frame average of the number of bases is outputted as the average value Nj* of the dimensionality (in a 
50 step S34). 

[0110] Next, the details of the sub-signal re-creation procedure (the step S4) shown in FIG. 5 will be explained with 
reference to FIG. 10. FIG. 10 is a flow chart showing the operation of the sub-signal re-creation section 3. 
[0111] Referring to FIG. 10, first, the sub-signal re-creation section 3 reads in a set of fixed length segments which 
is outputted from the initial sub-signal creation section 1 and a set of candidates for segmentation boundary which is 
55 outputted from the created sub-signal selection section 2 (in a step S42). Next, among the segmentation boundaries 
of the segments, attention is focused upon the top segmentation boundary (in a step S43), and the segmentation 
boundary is shifted to the top position among the segmentation boundary candidates for this top segmentation boundary 
(in a step S44). Next, for two segments which have a segmentation boundary in common, the dimensionality of the 
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compressed signal when the segmentation boundary is at the current position is calculated, and the average value 
with these being normalized by segment length is calculated (in a step S45). Note that, in the same manner as with 
the created sub-signal selection section 2, the average value of the dimensionality is calculated by the steps shown 
in FIGS. 8 and 9 and described above. 

[01 1 2] Next, if the average value of the dimensionality is the minimum value among those which have been calculated 
up till now for the segmentation boundary upon which attention is being focused (YES in a step S46), then this average 
value and the current position of the segmentation boundary are retained (in steps S47 and S48). If as yet the calculation 
has not been completed for all the candidates (NO in a step S49), then the segmentation boundary is shifted to the 
next candidate point (in a step S50) and the flow of control returns to the step S45, and the processes starting from 
the calculation of the average value of the dimensionality are repeated. When there are no further candidate points, 
the segmentation boundary upon which attention is being focused is shifted to the position of the segmentation bound- 
ary which corresponds to the minimum value of the average value of the dimensionality, and the segmentation boundary 
is definitely fixed upon. 

[0113] If the calculation has not been completed for the final segmentation boundary (NO in a step S51), then the 
segmentation boundary upon which the attention is focused is changed to the next segmentation boundary (in a step 
S52), and the flow of control returns to the step S44, and the processes up to here are repeated. When the operations 
have been completed for all the segmentation boundaries (YES in the step S51 ), then the sub-signal re-creation section 
3 outputs (in a step S53) the set of variable length segments which have been finally decided upon by shifting the 
segmentation boundary. 

[0114] Next, the details of the compression mapping determination procedure (the step S5) shown in FIG. 5 will be 
explained with reference to FIG. 11 . FIG. 11 is a flow chart showing the operation of the compression mapping deter- 
mination section 4. 

[0115] Referring to FIG. 11, first, the compression mapping determination section 4 reads in the set of segments 
which has been outputted from the sub-signal re-creation section 3 (in a step S55). Next, it extracts the base of each 
segment (in a step S56). Note that this extraction of the bases is performed, in the same manner as was performed 
by the created sub-signal selection section 2, according to the calculation steps which have been shown in FIG. 9 and 
described above. 

[0116] Next, the projection into the subspace is determined as the mapping for this segment (in a step S58). The 
compression mapping determination section 4 outputs the mapping which corresponds to each segment (in a step S59) . 
[0117] Next, the details of the signal compression procedure (the step S6) shown in FIG. 5 will be explained with 
reference to FIGS. 12 through 15. FIG. 12 is a flow chart showing the overall operation of the signal compression 
section 5. 

[0118] Referring to FIG. 12, first, the signal compression section 5 reads in the set of segments which has been 
outputted from the sub-signal re-creation section 3 and the set of linear mappings which has been outputted from the 
compression mapping determination section 4 (in a step S60). Next, using the set of sub-signals and the set of linear 
mappings which have been provided, a signal mapping procedure is performed (in a step S61) for projecting each of 
the histograms within the sub-signal into a subspace which has been created from that sub-signal. 
[0119] Furthermore, using the sets of sub-signals and of linear mappings which have been provided, and the set of 
compression histogram sequences which has been obtained by the signal mapping procedure, a projection distance 
calculation procedure is performed for calculating the distance between each histogram and the compression histogram 
which corresponds to it (in a step S62). Furthermore, using the set of the sequence of compression histograms which 
has been obtained by the signal mapping procedure and the projection distance which has been obtained by the 
projection distance calculation procedure, a compression feature creation procedure is performed for calculating a 
sequence of compression features (in a step S63). The signal compression section 5 outputs the sequence of com- 
pression features which has been obtained by this compression feature creation procedure (in a step S64). 
[0120] FIG. 13 is a flow chart showing the steps which are involved in the signal mapping procedure shown in FIG. 
12 (a step S61). 

[0121] Referring to FIG. 13, the signal mapping section 51 which is incorporated in the signal compression section 
5 first reads in the segment which is outputted from the sub-signal re-creation section 3 and the set of linear mappings 
which is outputted from the compression mapping determination section 4 (in a step S66). Next, it projects each his- 
togram within the segment into the subspace which is created from this segment (in a step S67). 
[01 22] In concrete terms, if the set of bases of the subspace which has been obtained from the segment X0> is taken 
as AV> = [a 1 U),a 2 V) r - t a N p>] T {j= 1 ,2,— } M) t then the compression histogram sequence 
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(/= 1 ,2,— is calculated in the following manner. 

5 

[0123] Here, Nj is the number of bases of the subspace which has been obtained from X(i), while XW is a matrix 
which is obtained by arranging Lj column vectors x<h, in other words 

F = [^,.,^] 

(J= 1,2,»-,M>. 

[0124] By doing this, the signal mapping section 51 outputs the set of compression histogram sequence Y< 1 ), Y( 2 \ ... 
Y( M > (in a step S68). 

15 [0125] FIG. 1 4 is a flow chart showing the steps which are involved in the projection distance calculation procedure 
shown in FIG. 12 (in a step S62). 

[0126] Referring to FIG. 14, the projection distance calculation section 52 which is incorporated in the signal com- 
pression section 5 first reads in the segment which is outputted from the sub-signal re-creation section 3, the set of 
linear mappings which is outputted from the compression mapping determination section 4, and the set of compression 
20 histogram sequence which is outputted from the signal mapping section 51 (in a step S70). Next, since the position of 
the compression histogram in the space in which the original histogram exists is obtained as described below, by 
inversely projecting the compression histogram (in a step S71 ), it calculates the distance between each histogram and 
the compression histogram which corresponds to it (in a step S72). 
[0127] In concrete terms, 

25 

X U) = A U?yU) + xU) where xU) = [ 5E U) ~U) r ..^U) ] u = l i2 ,-,M) 

30 

while each x, cy3 (/= 1 ,2,— ,Ly) gives the position in the original histogram space of the compression histogram y,G). 
[0128] Furthermore, the distance between xff* and is defined as the distance between the histogram and the 
compression histogram, and this is termed the projection distance of the histogram xfi. In other words, the projection 
distance of* is defined utilizing the Euclid distance as shown below. 



def r~n ~ 



where n is the dimensionality of the histogram, 



[0129] By doing this, the projection distance calculation section 52 outputs the projection distance which corresponds . 
to each histogram (in a step S73). 
so [0130] FIG. 15 is a flow chart showing the steps involved in the compression feature creation procedure (the step 
S63) shown in FIG. 12. 

[0131] Referring to FIG. 15, the compression feature creation section 53 which is incorporated in the signal com- 
pression section 5 first reads in the set of compression histogram sequence which has been outputted from the signal 
mapping section 51 and the projection distance which has been outputted from the projection distance calculation 
55 section 52 (in a step S75). Next, from the compression histogram y=(y 1t and the projection distance d(x,x ) 

which has been calculated in correspondence thereto, it creates a compression feature y* in the following manner (in 
a step S76): 
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5 where N is the dimensionality of the compression histogram y. 

[0132] By doing this, the compression feature creation section 53 outputs a sequence of compression features (in 
a step S77). 

- THE SECOND PREFERRED EMBODIMENT - 

10 

[0133] FIG. 2 is a block diagram showing the structure of the second preferred embodiment of the present invention. 
For this second preferred embodiment, an application to a signal retrieval device to which the signal compression 
device which was explained in the first preferred embodiment will be explained. In FIG. 2, structural elements to which 
the same reference symbols as those in the signal compression device of the first preferred embodiment which have 
15 been explained with reference to FIG. 1 , function in the same manner as in the first preferred embodiment, and thus 
the explanation thereof will be curtailed. 

[0134] Referring to FIG. 2, the reference symbol 6 denotes a reference feature extraction section which calculates 
a sequence of features from a reference signal, which is the signal to be retrieved. The reference symbol 7 denotes a 
stored feature extraction section which sets a window upon which attention is focused in a stored signal, which is an 

20 original signal which has been registered in advance, and which calculates a feature sequence from the signal within 
this window upon which attention is focused. The reference symbol 8 is a reference feature compression section which , 
based upon a mapping which has been outputted from the compression mapping determination section 4, compresses 
a reference feature sequence which has been outputted from the reference feature extraction section 6. The reference 
symbol 9 is a feature matching section which calculates the distance between the reference compressed signal which 

25 has been outputted from the reference feature compression section 8 and a stored compressed signal which has been 
outputted from the signal compression section 5 by newly using the feature sequence which has been outputted from 
the stored feature extraction section 7. The reference symbol 1 0 denotes a signal detection decision section which 
decides whether or not the reference signal is present in the location of the stored signal, by comparing the distance 
which has been outputted from the feature matching section 9 with a search threshold, which is a value corresponding 

30 to the distance. 

[01 35] The signal retrieval device shown in FIG. 2 takes as its input a reference signal, in other words a video signal 
which it is desired to retrieve for serving as a pattern or sample, and a stored signal, in other words a video signal 
which is retrieved, and outputs the location within the stored signal for which the distance from the reference signal is 
less than a value 6 1 which been set in advance (this is termed the search threshold). 
35 [0136] Next, the operation of the signal retrieval device shown in FIG. 2 will be explained with reference to FIGS. 1 6 
through 24. First, the overall operational flow of the signal retrieval device according to this second preferred embod- 
iment of the present invention will be explained with reference to the flow chart shown in FIG. 16, which shows the 
overall processing operation of this signal retrieval device. The detailed operation of each procedure will be explained 
hereinafter 

40 [0137] Referring to FIG. 16, first, the reference feature extraction section 6 performs a reference feature extraction 
procedure (in a step S81). Next, the stored feature extraction section 7 performs a stored feature extraction procedure 
(in a step S82). Next, the initial sub-signal creation section 1 performs an initial sub-signai creation procedure (in a 
step S83). Next, the created sub-signal selection section 2 performs a created sub-signal selection procedure (in a 
step S84). Next, the sub-signal re-creation section 3 performs a sub-signal re-creation procedure (in a step S85). Next, 

45 the compression mapping determination section 4 performs a compression mapping determination procedure (in a 
step S86). Next, the signal compression section 5 performs a signal compression procedure (in a step S87). Finally, 
the reference feature compression section 8 performs a reference feature compression procedure (in a step S88). 
[0138] Then, a window upon which attention is focused which is to be set for the stored compressed feature sequence 
which is outputted from the signal compression section 5 is set to the top of the stored signal (in a step S89). Next, the 

5 o feature matching section 9 performs a feature matching procedure (in a step S90). Furthermore, a signal detection 
decision section 10 performs a signal detection decision procedure (in a step S91). When performing this signal de- 
tection decision procedure, a decision is made (in a step S95) as to whether or not the current position of the window 
upon which attention is focused is at the end the stored signal; and, if the current position of the window upon which 
attention is focused is not at the end of the stored signal (NO in the step S95), then the window upon which attention 

55 is focused is shifted and the flow of control returns to the step S90 and the above described procedure is repeated. 
On the other hand, if the current position of the window upon which attention is focused is in fact at the end of the 
stored signal (YES in the step S95), then the search result for the signal is outputted (in a step S96). 
[0139] Note that the skip width calculation procedure of the step S92 which is shown in FIG. 1 6 is not necessary in 
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this second preferred embodiment, but will be explained in the third preferred embodiment, as described hereinafter. 
In the same manner, the distance re-calculation procedure of the step S93 and the signal detection re-decision pro- 
cedure of the step S94, which are shown in FIG. 1 6, are not necessary in this second preferred embodiment, but will 
be explained in the fourth preferred embodiment, as described hereinafter. 

[0140] Furthermore, since the above described initial sub-signal creation procedure (the step S83), the created sub- 
signal selection procedure (the step S84), the sub-signal re-creation procedure (the step S85), the compression map- 
ping determination procedure (the step S86), and the signal compression procedure (the step S87), are the same as 
the procedures which were executed by the signal compression device of the first preferred embodiment of the present 
invention shown in FIG. 6 through FIG. 15, accordingly the description thereof will be curtailed. However, the initial 
sub-signal creation procedure (the step S83) takes, as its input, the stored feature sequence which is outputted from 
the stored feature extraction procedure (the step S82). 

[01 41] Next, the details of the reference feature extraction procedure (the step S81 ) shown in FIG. 1 6 will be explained 
with reference to FIG. 1 7. FIG. 1 7 is a flow chart showing the operation of the reference feature extraction section 6. 
[0142] Referring to FIG. 17, first, the reference feature extraction section 6 reads in the reference signal which is 
provided (in a step S98). Next, the reference feature extraction section 6 performs feature extraction upon the reference 
signal which has been read in (in a step S99). 

[0143] Here, spectral features are utilized as the features which are to be extracted if the signal in question is an 
audio signal. For an audio signal, extraction of spectral features can be performed by band-pass filters. For example, 
if it is desired to retrieve an audio signal of about 1 5 seconds in length from a broadcast audio signal such as a television 
signal or a radio signal or the like, a good result may be obtained by ensuring that the concrete settings for feature 
extraction are as follows. Namely, if seven band-pass filters are utilized, and their central frequencies are set at equal 
intervals upon a logarithmic scale, and the average value of the square of the output of each band pass filter within 
the analysis window is calculated while shifting the analysis window of about 60 milliseconds in length by 1 0 milliseconds 
at a time, then the seven values which are obtained, when considered as a set, constitute a seven dimensional feature 
vector. In this case, a feature vector is obtained every 1 0 milliseconds. 

[0144] On the other hand, in the case of a video signal, color features are employed as the features. For example, 
if it is desired to retrieve a video signal of about 15 seconds in length from a broadcast signal such as a television 
signal or the like, a good result can be obtained by ensuring that the concrete settings for feature extraction are as 
follows. Namely, each of the images which constitute videos is segmented into two vertically and into three horizontally, 
and the RGB values are calculated for each segment, so that, when the three RGB values which have been obtained 
for each segment are grouped into a total of 1 8 values, they constitute an 1 8 dimensional feature vector. If the videos 
are built up from 30 frame images per second, then a single feature vector is obtained every 1/30th of a second. 
[0145] Next, by encoding the feature vectors using vector quantization (in a step S100), a histogram of the feature 
vectors is created from the time series of feature vectors (in a step S101). For example, if the number of codewords 
in the vector quantization is 512, the number of bins (partitions) in the entire histogram becomes 512, and each feature 
vector comes to be classified into one or the other of these 512 bins. Note that, in the following explanation, the his- 
togram which is created from the reference signal will be termed the reference histogram. The reference feature ex- 
traction section 6 outputs the reference histogram which has been obtained (in a step S102). 

[0146] Next, the details of the stored feature extraction procedure (the step S82) shown in FIG. 1 6 will be explained 
with reference to FIG. 18. FIG. 18 is a flow chart showing the operation of the stored feature extraction section 7. 
[0147] Referring to FIG. 1 8, first, the stored feature extraction section 7 reads in the stored signal (in a step S104). 
Next, for the stored signal which has been read in, the stored feature extraction section 7 sets (in a step S105) the 
window upon which attention is focused to the top of the stored signal. Here, the window upon which attention is focused 
is set to be of the same length as that of the reference signal which has been provided to the reference feature extraction 
section 6. 

[01 48] Next, feature extraction is performed upon the stored signal within the window upon which attention is focused 
(in a step S 106). Note that this feature extraction is performed by the same procedure as that which was performed 
by the reference feature extraction section 6. Furthermore, a histogram of the feature vector is created from the time 
series of the feature vector within the window upon which attention is focused (in steps S107 and S1 08). The way in 
which the histogram is created is the same as the method which was performed by the reference feature extraction 
section 6. The stored feature extraction section 7 repeatedly executes (in steps S109 and S110) the procedures from 
the step S1 06 to the step S1 08, while shifting the window upon which attention is focused (which was set to the top of 
the stored signal when starting the procedure) by one feature vector in order, until the end of the stored signal. Note 
that, in the following explanation, the histograms which have been created from the stored signal will be termed the 
stored histograms. Finally, the stored feature extraction section 7 outputs the stored histogram sequence which it has 
obtained (in a step S11 1 ). 

[0149] Next, the details of the reference feature compression procedure (the step S88) shown in FIG. 16 will be 
explained with reference to FIGS. 1 9 through 22. FIG. 1 9 is a flow chart showing the operation of the reference feature 
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compression section 8. 

[01 50] Referring to FIG. 1 9, first, the reference feature compression section 8 reads in (in a step S1 1 3) the reference 
histogram which is outputted from the reference feature extraction section 6 and the set of linear mappings which is 
outputted from the compression mapping determination section 4. Next, using each of the linear mappings, the refer- 
s ence feature compression section 8 performs a reference signal mapping procedure of projecting the reference histo- 
gram into the subspaces which corresponds thereto (in a step S114). This projection is performed by the same proce- 
dure as was performed by the signal compression section 5 and has been explained with reference to the first preferred 
embodiment of the present invention. For example, if the number of segments M is 1000, then 1000 compression 
histograms are created. 

10 [0151] Next, a reference projection distance calculation procedure is performed (in a step S115) of calculating the 
distance between the histogram and each compression histogram, in other words the projection distances of the his- 
togram. This calculation is performed by the same procedure as was performed by the signal compression section 5. 
Finally, a reference compression feature creation procedure is performed (in a step S116) of creating compression 
features from the compression histograms and the projection distances which correspond thereto. This creation of 

is compression features is performed by the same procedure as was performed by the signal compression section 5. 
The reference feature compression section 8 outputs (in a step S11 7) the set of reference compression features. 
[0152] FIG. 20 is a flow chart showing the steps involved in the reference signal mapping procedure shown in FIG. 
19 (the step S11 4). 

[0153] Referring to FIG. 20, a reference signal mapping section (not particularly shown in the figures) which is in- 
20 corporated in the reference feature compression section 8 first (in a step S1 1 9) reads in the reference histogram which 
is outputted from the reference feature extraction section 6 and the set of linear mappings which is outputted from the 
compression mapping determination section 4. Next, the reference signal mapping section projects the reference his- 
togram into a subspace according to the linear mapping which corresponds to each segment (in a step S120). 
[0154] By doing this, the reference signal mapping section outputs a set of reference compression histograms (in 
25 the step S1 21). 

[0155] FIG. 21 is a flow chart showing the steps involved in the reference projection distance calculation procedure 
shown in FIG. 19 (the step S 115). 

[0156] Referring to FIG. 21 , a reference projection distance calculation section (not particularly shown in the figures) 
which is incorporated in the reference feature compression section 8 first (in a step S123) reads in the reference 

30 histogram which is outputted from the reference feature extraction section 6, the set of linear mappings which is out- 
putted from the compression mapping determination section 4, and the set of reference compression histograms which 
is outputted from the reference signal mapping section. Next, by inversely projecting each compression histogram (in 
a step S124), the reference projection distance calculation section calculates the distance between each histogram 
and the compression histogram which corresponds thereto (in a step S125). 

35 [0157] By doing this, the reference projection distance calculation section outputs the projection distance which cor- 
responds to each histogram (in the step S126). 

[0158] FIG. 22 is a flow chart showing the steps involved in the reference compression feature creation procedure 
shown in FIG. 19 (the step S116). 

[0159] Referring to FIG. 22, a reference compression feature creation section (not particularly shown in the figures) 
40 which is incorporated in the reference feature compression section 8 f irst (in a step S1 28) reads in the set of reference 
compression histograms which has been outputted from the reference signal mapping section and the projection dis- 
tance which has been outputted from the reference projection distance calculation section. Next, the reference com- 
pression feature creation section creates a compression feature from the "compression histograms and the projection 
distance which has been calculated in correspondence therewith (in a step S129). 
45 [0160] By doing this, the reference compression feature creation section outputs a set of compression features (in 
the step S130). 

[0161] Next, the details of the feature matching procedure (the step S90) shown in FIG. 16 will be explained with 
reference to FIG. 23. FIG. 23 is a flow chart showing the operation of the feature matching section 9. 
[0162] Referring to FIG. 23, first, the feature matching section 9 reads in (in a step S132) the stored compressed 
50 feature sequence which is outputted from the signal compression section 5 and the set of reference compression 
features which is outputted from the reference feature compression section 8. Next, the feature matching section 9 
calculates the distance between the reference compression feature y* R and the stored compression feature y* s (in a 
step S133). 

[0163] In concrete terms, the distance d(y* R , y* s ) is defined as follows, using the Euclid distance. 

55 
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E0> M-ysi) 



= -id{y R ,y s f +{d(x Jt ,Xg)-d(x s ,x s )} 2 ■• 



(2) 



J0 



where x R is the reference histogram, x s is the stored histogram, and y R and y s are the compression histograms 
which correspond to x R and x s . x R and 5? s are the positions of y R and y s in the histogram space, while y* RJ and y* Si 
are the i-th dimension values of y* R and y* s respectively. 

[0164] Here, due to the properties of the KL transform (the principal component analysis), the following Equation 
holds: 



15 



d(y R ,y s ) = d(x R ,x s )<d(x R ,x s ) ...(3) 



[0165] Furthermore, d(y* R , y* s ) has the following property: 

20 

d(x R ,x s ) < d(y R9 y s ) < d(y* Ri y/) 

= mm{d(x R ,x s )}<d(x Ry x s ) --.(4) 

25 

[0166] The minimum values in Equation (4) are calculated for all the groups (xr, x s ) of the histogram when y Rf ys> d 
( x r*x.r) and d(x s ,?s) ar e g |v ©n- 

[0167] Furthermore, as can be understood from Equation (4), due to the property expressed by Equation (3), the 
principal component analysis produces a unique result that the distance value between compression features becomes 
30 the lower limit value of the distances between the histograms. In addition, by further using the projection distance, it 
is possible to obtain a larger lower limit value d(y* R , y* s ) for the distance between the histograms, as compared to the 
case in which it is not used. 

[0168] In this manner, the feature matching section 9 outputs the lower limit value of the distance which has been 
obtained (in a step S134), 

35 [01 69] Next, the details of the signal detection decision procedure (of the step S91 ) shown in FIG. 1 6 will be explained 
with reference to FIG. 24. FIG. 24 is a flow chart showing the operation of the signal detection decision section 10. 
[0170] Referring to FIG. 24, first, the signal detection decision section 1 0 reads in the distance lower limit value which 
is outputted from the feature matching section 9 (in a step S1 39). Next, the signal detection decision section 1 0 com- 
pares together (in a step S1 40) the distance lower limit value and a search threshold which is a value which is deter- 

40 mined in advance based upon the Euclid distance, which is the distance measure. If the distance value is less than 
the search threshold (i.e. if, when the window upon which attention is focused has been segmented in the time direction, 
it is detected that the distance value for all the time segments is less than the search threshold) (YES in the step S1 40), 
then it is decided (in a step S141) that the reference signal exists in the location of the stored signal, and the current 
position in time series with respect to the stored signal (a flag) is outputted (in a step S1 42) as the signal detection result. 

45 

- THE THIRD PREFERRED EMBODIMENT - 

[0171] FIG. 3 is a block diagram showing the structure of a third preferred embodiment of the present invention. In 
the description of this third preferred embodiment, a signal retrieval device will be explained, in which, to the signal 

50 retrieval device which has been explained in the description of the second preferred embodiment, there is newly pro- 
vided a skip width calculation section 11 which, based upon the distance which has been outputted from the feature 
matching section 9, calculates the skip width of the window upon which attention is focused which is set with respect 
to the stored compressed feature sequence which is outputted from the signal compression section 5, and which shifts 
the window upon which attention is focused by that skip width. In this FIG. 3, since structural elements to which the 

55 same reference numerals as ones in the signal compression device of the first and second preferred embodiments 
which have been explained with reference to FIG. 1 and ones in the signal retrieval device which has been explained 
with reference to FIG. 2 are appended, function in the same manner as in the signal compression device of the first 
preferred embodiment or the signal retrieval device of the second preferred embodiment, and the explanation thereof 
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will be curtailed. 

[01 72] Note that the signal retrieval device shown in FIG. 3 also takes as its input a reference signal, in other words 
a video signal which it is desired to retrieve for serving as a pattern or sample, and a stored signal, in other words a 
video signal which is retrieved, and outputs the location within the stored signal for which the distance from the reference 

5 signal is less than a value which been set in advance (this is termed the search threshold). 

[0173] Next, referring to FIGS. 1 6 and 25, the operation of the signal retrieval device shown in FIG. 3 will be explained. 
First, to explain the overall operational flow of this signal retrieval device with reference to FIG. 1 6, a feature of the 
operation of the signal retrieval device of this third preferred embodiment of the present invention is that, by contrast 
to the flow of operation which was explained above in connection with the second preferred embodiment, between the 

10 "signal detection decision procedure" of the step S91 and the "procedure of determining whether or not the current 
position of the window upon which attention is focused is the end of the stored signal" of the step S95, a skip width 
calculation procedure is executed (in a step S92) in which the skip width is calculated of the window upon which 
attention is focused which is set for the stored compressed feature sequence which is outputted from the signal com- 
pression section 5, and the window upon which attention is focused is shifted by this skip width. 

is [0174] Next, the details of the skip width calculation procedure (the step S92) shown in FIG. 16 will be explained 
with reference to FIG. 25. FIG: 25 is a flow chart showing the operation of the skip width calculation section 11 . 
[0175] Referring to FIG. 25, first, the skip width calculation section 11 reads in the distance lower limit value which 
is outputted from the feature matching section 9 (in a step S144). Next, the skip width calculation section 11 calculates 
(in a step S1 45) the skip width over which it is possible to eliminate feature matching, in other words distance calculation 

20 while still ensuring that false dismissal does not occur. 

[0176] The principle by which the skip width is determined will be explained hereinafter. 

[0177] Since the histograms are ones which have been obtained by classifying time series of feature vectors and 
accumulating them, the distance value between histograms does not change abruptly in accompaniment with shifting 
of the time window with respect to the feature vector in the stored signal. The absolute value of the change ratio of the 
25 distance value with respect to shifting of the time window by one feature vector certainly does not exceed V2. In other 
words, if the distance value between the histograms when the top of the time window upon stored signal is the m^th 
feature vector is termed d(x Rl x s (m.,)), then the lower limit of the distance value df k x R ,x^m 2 )) when the time window 
has been shifted to the m 2 -th feature vector and when m 1 <rn 2 <(m 1 +D), is given by the following equation: 

30 

d(x R ,x s (m 2 )) = d(x Ri x<£mjyj2{m 2 - m,) (5) 



35 



45 



where D is the width of the time window. Due to Equation (4) above, Equation (5) is changed as follows: 



def 



4.(x R ,x s (m 2 ))>d\y Jt9 y s (m l ))-42(m 2 -m l ) = d* 



•(6) 



[0178] Since the distance value is not less than zero, when the lower limit value cT given by Equation (6) is less than 
zero, then "0" becomes the lower limit value. By replacing the lower limit value with the search threshold and m 2 - 
m 1 with the skip width o>, it is possible to obtain a skip width in the following manner: 



so 



0) = 



floor( ( ^v/ l) )+1 (if -* >6 ^ 

1. (otherwise) 



where ffoorfx) is the largest integer which does not exceed x. 
[01 79] Although, when starting this procedure, the stored compression feature is extracted from the top of the com- 
pressed feature sequence, during the progress of the procedure, the procedure is continued while shifting the position 
from which the stored compression feature is extracted along the time domain in order (in a step S146). The amount 
by which such shifting along the time domain is performed is provided by the skip width calculation section 11 . 



27 



EP 1 426 899 



Page 28 of 103 



EP 1 426 899 A2 

-THE FOURTH PREFERRED EMBODIMENT - 

[0180] FIG. 4 is a block diagram showing the structure of a fourth preferred embodiment of the present invention. In 
the description of this fourth preferred embodiment, a signal retrieval device will be explained, in which, to the signal 

5 retrieval device which has been explained in the description of the third preferred embodiment, there are newly provided 
a distance re-calculation section 12 which, for the place in the stored signal for which it has been decided by the signal 
detection decision section 1 0 that the reference signal is present, calculates the distance between the feature sequence 
which has been outputted from the reference feature extraction section 6, and the feature sequence which has been 
outputted from the stored feature extraction section 7, and a signal detection re-decision section 1 3 which, by comparing 

10 together the distance which has been outputted from the distance re-calculation section 12 and a search threshold, 
decides for a second time whether or not the reference signal is present at the location in the stored signal. In this FIG. 
4, since structural elements to which the same reference symbols as ones in the signal compression devices or signal 
retrieval devices of the first through the third preferred embodiments and which have been explained with reference 
to FIGS. 1 through 3 are appended, function in the same manner as in these signal compression devices or signal 

is retrieval devices, the explanation thereof will be curtailed. 

[0181] Note that the signal retrieval device shown in FIG. 4 also takes as its input a reference signal, in other words 
a video signal which it is desired to retrieve for serving as a pattern or sample, and a stored signal, in other words a 
video signal which is retrieved, and outputs the location within the stored signal for which the distance from the reference 
signal is less than a value e 1 which been set in advance (this is termed the search threshold). 

20 [0182] Next, the operation of the signal retrieval device which is shown in FIG. 4 will be explained with reference to 
FIGS. 1 6, 26, and 27. First, to explain the overall operational flow of this signal retrieval device with reference to FIG. 
1 6, a feature of the operation of the signal retrieval device of this preferred embodiment of the present invention is that, 
by contrast to the flow of operation which was explained above in connection with the third preferred embodiment, 
between the "skip width calculation procedure" of the step S92 and the "procedure of determining whether or not the 

25 current position of the window upon which attention is focused is the end of the stored signal" of the step S95, there 
are executed a distance re-calculation procedure (in a step S93) in which, for the location in the stored signal for which 
it has been decided by the signal detection decision section 10 that the reference signal is present, the distance is 
calculated between the feature sequence which has been outputted from the reference feature extraction section 6 
and the feature sequence which has been outputted from the stored feature extraction section 7, and a signal detection 

30 re-decision procedure (in a step S94) in which, by comparing together the distance which has been obtained by the 
above distance re-calculation procedure and the search threshold, a decision is again made as to whether or not the 
reference signal is present in the location in the stored signal. 

[01 83] Next, the details of the distance re-calculation procedure (of the step S93) shown in FIG. 1 6 will be explained 
with reference to FIG. 26. FIG. 26 is a flow chart showing the operation of the distance re-calculation section 12. 

35 [0184] Referring to FIG. 26, first, the distance re-calculation section 12 reads in the reference histogram which is 
outputted from the reference feature extraction section 6, the stored histogram sequence which is outputted from the 
stored feature extraction section 7, and the detection result which is outputted from the signal detection decision section 
10 (in a step S148). Next, the distance from the reference histogram is calculated for the stored histogram which 
corresponds to the location within the stored signal at which it has been decided that the reference signal is present 

40 (in a step S1 49). The distance between these histograms is defined in the same manner as in Equation (1) described 
previously, using the Euclid distance. The distance value which has been obtained is then outputted (in a step S150). 
[0185] Next, the details of the signal detection re-decision procedure (the step S94) shown in FIG. 1 6 will be explained 
with reference to FIG. 27. FIG. 27 is a flow chart showing the operation of the signal detection re-decision section 13. 
[0186] Referring to FIG. 27, first, the signal detection re-decision section 13 reads in the distance value which is 

45 outputted from the distance re-calculation section 12 (in a step S152). Next, the signal detection re-decision section 
13 compares together the distance value and the search threshold (in a step S153). If the result of this comparison is 
that the distance value is less than the search threshold (YES in the step S153), then, since this means that this 
reference signal is present within the stored signal, the signal detection re-decision section 13 outputs (in a step S1 54) 
the current position in time series with respect to the stored signal as the signal detection result. 

50 [0187] Note that, according to requirements, the skip width calculation section 11 may, or may not, be included in 
the signal retrieval device shown in FIG. 4. 

- RESULTS OF ACTUAL EXPERIMENTS - 

55 [0188] Next, actual experimental results which were obtained during the operation of a signal retrieval device ac- 
cording to the present invention will be explained. 

[01 89] In order to check upon the beneficial effects of the present invention, as a first experiment, a histogram which 
was made from 24 hours of video signal was taken as a stored signal, and investigation was performed of change in 
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the average dimensionality of a compression histogram which was normalized by segment length, when the number 
of segments and the contribution threshold were varied. 

[0190] Note that the parameters of the compression were: sampling frequency = 29.97 Hz; number of segments of 
the image = 6 (two segments vertically and three segments horizontally); dimensionality of the histogram = 256; and 
5 width of the time window = 15 seconds. Furthermore, the segmentation boundary shiftable width A was varied from 1 
up to 500 in the following manner: 1 , 2, 5, 10 

[0191] The results of this first experiment are shown in FIG. 28. The segmentation boundary shiftable width A (termed 
"Width of Shiftable Range" in FIG. 28) is shown along the horizontal axis of the graph, while the ratio of the average 
dimensionality to the average dimensionality when A = 0 as a reference (termed "Ratio of Dimensions" in FIG. 28) is 
10 shown along its vertical axis. For example, for a segmentation boundary shiftable width A = 500, number of segments 
M = 1000 (segments), and contribution threshold (contribution rate) o = 0.75, the average dimensionality was 2.91 ; 
while when the segmentation boundary shiftable width A was equal to 0, the average dimensionality was 3.30 and the 
dimensionality reduction ratio was 0.882. 

[0192] Next, as a second experiment, a histogram which was made from 24 hours of video signal was taken as an 
15 original signal, and investigation was performed of the number of times the average dimensionality was calculated by 
the created sub-signal selection section 2 and the sub-signal re-creation section 3. Note that the search parameters 
were the same as for the first experiment, and moreover the number of segments M was = 1 000 (segments) and the 
contribution threshold (contribution rate) o was = 0.75. 

[0193] The results of this second experiment are shown in FIG. 29. The segmentation boundary shiftable width A 
20 (termed "Width of Shiftable Range" in FIG. 29) is shown along the horizontal axis of the graph, while the number of 
times the calculation was performed (termed "Number of Calculation" in FIG. 29) is shown along its vertical axis. For 
example, for a segmentation boundary shiftable width A = 500, the number of times the calculation was performed with 
the method of the present invention (shown by "speedup" in the graph of FIG. 29) was about 80,000, while the number 
of times the calculation was performed with a method in which no created sub-signal selection section 2 was imple- 
25 mented (shown by "non-speedup" in the graph of FIG. 29) was about 1 ,000,000; so that the reduction ratio of calculation 
number attained by the practice of the present invention was about 12.5. 

-THE FIFTH PREFERRED EMBODIMENT - 

30 [0194] Next, a fifth preferred embodiment of the present invention will be explained with reference to the figures. 
[0195] In the description of this fifth preferred embodiment, an audio signal will be employed as an example of the 
object signal which is to be processed. Note that the concrete feature extraction method and method of creating the 
histograms will be explained hereinafter in the context of the processing which is performed by a query feature extraction 
section 101. 

35 [0196] FIG. 30 is a functional block diagram showing the structure of a signal retrieval device to which the method 
of this fifth preferred embodiment of the present invention has been applied. Referring to FIG. 30, the signal retrieval 
device of this preferred embodiment comprises: a query feature extraction section (a reference feature extraction sec- 
tion) 101 which extracts, features from a query signal (a reference signal); a database feature extraction section (a 
stored feature extraction section) 102 which sets a window upon which attention is focused in a database signal (a 

40 stored signal), and extracts features from that signal within the window upon which attention is focused; a database 
feature partitioning section 103 which partitions feature sequences which are outputted by repeatedly performing the 
processing of the database feature extraction section 102 while shifting the window upon which attention is focused; 
a database feature pruning section 104 which extracts representative features from the feature sequences after par- 
titioning which have been outputted from the database feature partitioning section 1 03, and which produces represent- 

45 ative feature sequences which are created with a smaller number of features; a feature region extraction section 1 05 
which produces regions in which features are present which are included in the partitions which have been outputted 
from the database feature partitioning section 1 03; a feature matching section 106 which calculates the distance be- 
tween the feature sequences which have been outputted from the query feature extraction section 1 01 and the repre- 
sentative feature sequences which have been outputted from the database feature pruning section 104; a distance 

50 compensation section 1 07 which compensates the distance which has been outputted from the feature matching sec- 
tion 1 06 by using the regions which have been outputted from the feature matching section 106; and a signal detection 
decision section 108 which, by comparing together the distance which has been outputted from the distance compen- 
sation section 107 and a search threshold, which is a threshold which corresponds to the distance, decides whether 
or not the query signal is present at the location within the database signal. 

55 [0197] The signal retrieval device shown in FIG. 30 takes as its input a query signal, in other words an audio signal 
which it is desired to retrieve for serving as a pattern or sample, and a database signal, in other words an audio signal 
which is retrieved, and outputs the location within the database signal for which the distance from the query signal is 
less than a value which been set in advance (this is termed the search threshold). 
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[0198] Next, the operation of the signal retrieval device which is shown in FIG. 30 will be explained with reference 
to FIGS. 30 and 38 through 42. Note that the procedures which are performed by the query feature extraction section 
101 , the database feature extraction section 102, and the signal detection decision section 108 described above are 
the same as in the case of the second preferred embodiment which was described above with reference to FIGS. 1 6 
s through 24. 

[0199] FIG. 38 is a flow chart showing the processing which is performed by the database feature partitioning section. 
[0200] Next, referring to FIG. 3B, the database feature partitioning section 103 reads in the sequence (the feature 
sequence) of the DB histogram (the stored histogram) which is outputted by repeating the processing of the database 
feature extraction section 102 while shifting the window upon which attention is focused (in a step S164). Next, the 
10 • database feature partitioning section 103 partitions the histogram sequence (in a step S1 65). Although various types 
of method for this partitioning may be considered, here two such methods will be explained. 

[0201] In the first such method, the histogram sequence is segmented equally according to a segmentation width 
which is specified in advance. For example, when the segmentation width is taken as 50, a large number of partial 
histogram sequences (partial feature sequences) are created, each of length 50 frames. In the second such method, 

15 the lengths of the partial histogram sequences are adjusted so that each histogram of the partial histogram sequence 
is present within a fixed distance from a histogram representing the partial histogram sequence, which is produced by 
the feature region extraction section 1 05 which will be described hereinafter. In concrete terms, the following procedure 
is performed. First, the partial histogram sequence is set to a certain length, and a representative histogram is extracted 
from among the sequence. The method for choosing the representative histogram is according to the procedures 

20 performed by the database feature pruning section 104. Next, the distance from the representative histogram is cal- 
culated for each histogram within the partial histogram sequence, and its maximum value is obtained. The above 
described operation is repeated while increasing the length of the partial histogram sequence one unit at a time, from 
a start length for the partial histogram sequence of 1 until the maximum value exceeds a region threshold which has 
been set in advance. The above is the second method for partitioning the histogram. The database feature partitioning 

25 section 1 03 then outputs (in a step S1 66) the set of partial histogram sequences which have thus been produced by 
segmentation. 

[0202] FIG. 39 is a flow chart showing the procedure which is performed by the database feature pruning section 
104. Referring to this FIG. 39, first, the database feature pruning section 104 reads in the set of partial histogram 
sequences which is outputted from the database feature partitioning section 1 03 (in a step S1 67). Next, the database 

30 feature pruning section 1 04 extracts a representative feature from each partial histogram sequence (in a step S1 68). 
Although various types of method may be considered for this extraction of representative features, in this specification, 
two such methods will be explained. In the first such method, some one of the histograms within the partial histogram 
sequence is taken as a representative feature thereof, just as it is. For example, the histogram at the top of the partial 
histogram sequence is taken as being representative thereof. In the second method, the centroid of the histograms 

35 (with reference to the Euclid distance space) within the partial histogram sequence is calculated, and this is taken as 
being representative thereof . Finally, the representative features are arranged in order of their original partial histogram 
sequences, and the result is taken as being a new sequence (in a step S1 69). The feature sequence which has been 
produced according to these steps is termed the representative feature sequence. Then, the database feature pruning 
section 104 outputs this representative feature sequence (in a step S170). 

40 [0203] FIG. 40 is a flow chart showing the processing which is performed by the feature region extraction section 1 05. 
[0204] Next, referring to FIG. 40, the feature region extraction section 1 05 reads in the set of partial histogram se- 
quences which is outputted from the database feature partitioning section 1 03 and the representative feature sequence 
which is outputted from the database feature pruning section 1 04 (in a step S1 71 ). Next, for each histogram within the 
partial histogram sequence, the feature region extraction section 105 calculates the distance from a representative 

45 feature, and calculates the maximum value dmax thereof (in a step S172). By the above described operation, it is 
possible to determine a range within the partial histogram sequence in which a histogram is present, and this comes 
within the range of the distance dmax from the representative feature. The feature region extraction section 105 then 
outputs a set of the dmax (in a step S1 73). 

[0205] FIG. 41 is a second flow chart showing the processing which is performed by the feature matching section 106. 

so [0206] Next, referring to FIG. 41, the feature matching section 106 reads in (in a step S 174) the representative 
feature sequence which is outputted from the database feature pruning section 104 and the query feature which is 
outputted from the database feature extraction section 102. Next, the feature matching section 106 calculates the 
distance between the query feature Xq and the representative feature x D (in a step S1 75). Although various distance 
measures might be used forthis calculation, for example, the Manhattan distance or the Euclid distance may be used. 

55 [0207] The Manhattan distance is defined as 
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def „ 




5 



while the Euclid distance is defined as 



10 



def 



f5 



[0208] Then, the feature matching section 1 06 outputs (in a step S 1 76) the distance d(XQ,x D ). 

[0209] FIG. 42 is a flow chart showing the processing which is performed by the distance compensation section. 

20 [0210] Next, referring to FIG. 42, the distance compensation section 1 07 reads in (in a step S1 77) the distance value 
dmax which is outputted from the feature region extraction section 105 and the distance value d(XQ, x D ) which is 
outputted from the feature matching section 106. Next, the distance compensation section 107 calculates the com- 
pensated distance value. In other words, the distance compensation section 107 calculates d(XQ, x D ) - dmax, thus 
correcting the distance value (in a step S178). By doing this, for the partial histogram sequence which corresponds to 

25 the representative feature, the minimum value of the distance from the histograms therein is obtained. The distance 
compensation section 107 then outputs (in a step S179) the distance value after compensation (the compensated 
distance value). 

- THE SIXTH PREFERRED EMBODIMENT - 



[0211] Next, a sixth preferred embodiment of the present invention will be described with reference to the drawings. 
[0212] FIG. 31 is a functional block diagram showing a signal retrieval device to which the method of the sixth pre- 
ferred embodiment has been applied. With the signal retrieval device of this preferred embodiment, in the signal retrieval 
device according to the fifth preferred embodiment of the present invention, there are further comprised a segment 

35 extraction section 109, a compression mapping determination section 110, a database feature compression section 
(a stored feature compression section) 111, and a query feature compression section (a reference feature compression 
section) 112; and it takes as its input a query signal, in other words an audio signal which it is desired to retrieve for 
serving as a pattern or sample, and a database signal, in other words an audio signal which is retrieved, and it outputs 
the location within the database signal for which the distance from the query signal is less than a value G 1 which been 

40 set in advance (this is termed the search threshold). 

[0213] The segment extraction section 109, by segmenting the feature sequences which the database feature ex- 
traction section 1 02 has produced by repeatedly shifting the window upon which attention is focused, extracts segments 
(sub-signals) which are partial sequences. Furthermore the compression mapping determination section 110 deter- 
mines, from each segment which has been outputted from the segment extraction section 109, upon a mapping for 

^5 calculating features of lower dimensions than the features. Based upon the mapping which has been outputted from 
the compression mapping determination section 1 1 0, the database feature compression section 1 1 1 calculates features 
which correspond to the segments which have been outputted from the segment extraction section 109, and which 
are of lower dimensions than the features. Moreover, based upon the mapping which has been outputted from the 
compression mapping determination section 110, the query feature compression section 112 calculates features which 

so correspond to the features which have been outputted from the query feature extraction section 1 01 , and which are of 
lower dimensions than the features. 

[0214] Next, the procedures which are performed by the signal retrieval device of this sixth preferred embodiment 
will be explained in detail with reference to FIGS. 31 and 43. 

[0215] First, the procedures which are performed by the query feature extraction section 1 01 , the database feature 
S5 extraction section 1 02, the compression mapping detemni nation section 1 1 0, the database feature compression section 
111 , and the query feature compression section 112 are the same as in the case of the second preferred embodiment 
which was described above, and accordingly their explanation will be curtailed. 

[0216] FIG. 43 is a flow chart showing the processing which is performed by the segment extraction section 1 09. 
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[021 7] The segment extraction section 1 09 may have either one of two different structures. The first structure is that 
this segment extraction section 109 only incorporates the initial sub-signal creation section 1 described above. The 
second structure is that this segment extraction section 1 09 comprises the initial sub-signal creation section 1 , the 
created sub-signal selection section 2, and the sub-signal re-creation section 3. in either case, since the procedures 

5 which are performed by these sections are the same as in the case of the second preferred embodiment which was 
described above (in steps S 1 83 through S 1 84), accordingly their explanation will be curtailed. The segment extraction 
section 1 09 outputs (in a step S 1 85) a set of segments which are histogram sequences which have been segmented. 
[021 6] After this, the operations of the database feature partitioning section 1 03 through the signal detection decision 
section 108 are performed. Note that, since the procedures which are performed by the database feature partitioning 

w section 103 through the signal detection decision section 108 are the same as in the case of the fifth preferred em- 
bodiment which was described above, accordingly their explanation will be curtailed. 

- THE SEVENTH PREFERRED EMBODIMENT - 

is [0219] Next, a seventh preferred embodiment of the present invention will be explained with reference to the draw- 
ings. 

[0220] FIG. 32 is a functional block diagram showing the structure of a signal retrieval device to which the method 
of this seventh preferred embodiment of the present invention has been applied. With the signal retrieval device of this 
preferred embodiment, in the signal retrieval device according to the sixth preferred embodiment of the present inven- 

20 tion, there are further comprised a distance re-calculation section 1 1 3 and a signal decision re-calculation section 1 1 4; 
and it takes as its input a query signal, in other words an audio signal which it is desired to retrieve for serving as a 
pattern or sample, and a database signal, in other words an audio signal which is retrieved, and it outputs the location 
within the database signal for which the distance from the query signal is less than a value e 1 which been set in advance 
(this is termed the search threshold). 

25 [0221] Here, for the location in the database signal at which it has been decided by the signal detection decision 
section 108 that the query signal is present, the distance re-calculation section 113 calculates the distance between 
the feature sequence which has been outputted from the query feature extraction section 1 01 and the feature sequence 
which has been outputted from the database feature extraction section 1 02. In addition, the signal detection re-decision 
section 114 re-determines whether or not the query signal is present at the location in the database signal by comparing 

30 together the distance which has been outputted from the distance re-calculation section 1 1 3 and the search threshold. 
[0222] Since the other procedures are the same as in the case of the fourth preferred embodiment which was de- 
scribed above, their explanation will be curtailed. 

-THE EIGHTH PREFERRED EMBODIMENT - 

35 

[0223] Next, an eighth preferred embodiment of the present invention will be described in detail with reference to 
the figures. 

[0224] FIG. 33 is a figure showing functional blocks of a signal retrieval device to which a method according to the 
eighth preferred embodiment of the present invention has been applied. The signal retrieval device of this eighth pre- 

*o f erred embodiment is constructed by adding a skip width calculation section 11 8 to the signal retrieval device of the 
seventh preferred embodiment of the present invention, and takes as its input a query signal, in other words an audio 
signal which has been retrieved and which serves as a pattern or sample, and a database signal, in other words an 
audio signal which is retrieved, and outputs a location in the database signal at which the distance from the query 
signal is less than a value 9 1 which has been set in advance (this is termed the search threshold). 
[0225] Here, the skip width calculation section 118 calculates a skip width for the window upon which attention is 
focused which is set for the compressed feature sequence which is outputted from the database feature compression 
section 111 based upon the distance which has been outputted from the feature matching section 106, and shifts the 
window upon which attention is focused by this skip width. This procedure is the same as that performed in the case 
of the third preferred embodiment of the present invention described above. The other procedures of this eighth pre- 

50 f erred embodiment are the same as those performed in the case of the seventh preferred embodiment of the present 
invention and described above, and accordingly their description will be curtailed. Note that, according to requirements, 
the distance re-calculation section 113 and the signal detection re-decision section 114 may be included in the signal 
retrieval device shown in FIG. 33, or, if they are not required, may not be included. 

55 - RESULTS OF ACTUAL EXPERIMENTS - 

[0226] Next, actual experimental results which were obtained during the operation of a signal retrieval device ac- 
cording to the eighth preferred embodiment of the present invention will be explained. 
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[0227] An audio signal 200 hours long was taken as the stored signal, and signal retrieval was performed for 200 
reference signals, each consisting of a different 15 seconds which had been selected randomly from a signal which 
was provided separately from the stored signal; and the time period which was required for retrieval from when the 
reference signal was inputted, and the number of times the processing of the feature matching section 106 was exe- 

5 cuted, in other words the number of feature matches, were investigated. Note that, at this time, the parameters were 
as follows: dimensionality of the feature vectors = 7; time width of the feature vectors = 60 msec; time resolution of the 
feature vectors = 1 0 msec; dimensionality of the histograms = 128. The compression features were created for each 
a = 50 frames. Furthermore, the contribution threshold was = 0.9, and the search threshold was ■ 85. 
[0228] The experimental results which were obtained for this preferred embodiment are shown in FIGS. 48 and 49. 

10 FIG. 48 is a third explanatory figure showing the experimental results. FIG. 49 is a fourth explanatory figure showing 
the experimental results. 

[0229] The number of segments is, in each case, shown along the horizontal axis; and in FIG. 48 the time period 
which was required for the retrieval is shown along the vertical axis, while in FIG. 49 the number of matches is shown 
along the vertical axis. When the number of segments was taken as 1 0,000, with the method of the present invention 

15 ("proposed method"), the time period which was required for the retrieval was 0.364 seconds and the number of matches 
was 772,784; while, with a method in which matching was performed with out taking advantage of the projection distance 
("projection distance unavailable"), the time period which was required for the retrieval was 1 .491 seconds and the 
number of matches was 1 ,036,493; and, with a conventional method (i.e. with a conventional method as described in 
Japanese Patent No. 3,065,314), the time period which was required for the retrieval was 4.218 seconds and the 

20 number of matches was 633,047. 

- THE NINTH PREFERRED EMBODIMENT - 

[0230] Next, a ninth preferred embodiment of the present invention will be explained with reference to the drawings, 

25 [0231] FIG. 34 is a functional block diagram showing the structure of a signal retrieval device to which the method 
of this ninth preferred embodiment of the present invention has been applied. With the signal retrieval device of this 
preferred embodiment, in the signal retrieval device according to the seventh preferred embodiment of the present 
invention , there are further comprised a database feature classification section 1 1 5, a selection threshold setting section 
116, and a database feature selection section 117; and the device takes as its input a query signal, in other words an 

30 audio signal which it is desired to retrieve for serving as a pattern or sample, and a database signal, in other words an 
audio signal which is retrieved, and the device outputs the location within the database signal for which the distance 
from the query signal is less than a value 8., which been set in advance (this is termed the search threshold). 
[0232] The database feature classification section 115 classifies each feature which has been produced in the 
processing which has been performed by the database feature extraction section 1 02 by repeating its operation while 

35 shifting the widow upon which attention is focused, based upon a distance which has been defined in advance, and 
determines upon a representative feature of the classified features. The selection threshold setting section 1 1 6 calcu- 
lates a selection threshold for the distance which has been defined by the database feature classification section 1 1 5 
from the search threshold which has been defined in advance. Furthermore, among the classifications which have 
been outputted from the database feature classification section 1 1 5, the database feature selection section 1 1 7 selects 

40 a feature included in the classification which is such that the distance from the feature which has been outputted from 
the query feature extraction section 101 satisfies the condition which has been produced from the selection threshold 
which has been outputted from the selection threshold setting section 116. 

[0233] Next, the operation of the signal retrieval device of this preferred embodiment of the present invention will be 
explained with reference to FIG. 34 and FIGS. 44 through 47. First, since the procedures which are performed by the 

45 query feature extraction section 1 01 and the database feature extraction section 1 02 are the same as in the case of 
the seventh preferred embodiment which was described above, accordingly their explanation will be curtailed. 
[0234] FIG. 44 is a flow chart showing the processing which is performed by the database feature classification 
section. Referring to this figure, the database feature classification section 1 1 5 reads in (in a step S21 0) the histogram 
sequences which have been outputted by the database feature extraction section 1 02 repeatedly performing its oper- 

so ation while shifting the widow upon which attention is focused. Next, the database feature classification section 115 
classifies each of the histograms in this histogram sequence according to, for example, the Euclid distance (in a step 
S211). This classification of the histograms is performed by regarding each of the histograms as a vector having the 
dimensionality which is equal to the number of its bins, and by encoding these vectors utilizing vector quantization. 
For example, if the number of codewords employed for the vector quantization is 1 024, then each of the histograms 

55 comes to be classified, into some one of the 1 024 sets (these are termed clusters). Each of the clusters is considered 
as being represented by a histogram (termed the centroid histogram) which is the centroid of the histograms which 
belong to that cluster. At this time, the clusters are constituted so that the sum of the distances between the histograms 
which belong to each of the clusters and Its centroid histogram is a minimum, and moreover so that, for each of the 
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histograms which belongs to each of the clusters, its distance from the centra id histogram of the cluster to which it 
belongs comes to be smaller than the distance from the centroid histogram of any other cluster. The database feature 
classification section 115 then outputs (in a step S212) the set of clusters which are the classifications of the histograms. 
[0235] FIG. 45 is a flow chart showing the processing which is performed by the selection threshold setting section. 

s [0236] Next, referring to FIG. 45, the selection threshold setting section 116 reads in the search threshold 8., (in a 
step S213), and calculates a selection threshold 6 2 from this search threshold 9 1 (in a step S214). Here by the selection 
threshold is meant, when a cluster is selected which has a possibility of including the DB histogram which corresponds 
to the signal to be retrieved, an upper limit for the distance between the query histogram and the cluster. If the distance 
measure which is employed when performing the matching is Euclid distance, then the selection threshold is the same 

10 as the search threshold. Since the measures which are employed for classification and for selection are mutually 
compatible, there are no cases of the occurrence of false dismissal. If the distance measure which is employed when 
performing the matching is Manhattan distance, then the selection threshold is set using the parameterp in the following 
manner: 



[0237] The selectable region of the cluster becomes narrower as p is set larger. When p = 0, the maximum value of 
20 the region is obtained for which it is theoretically possible to guarantee that no false dismissal will occur. Since actually 
hardly any false dismissals occur even if p is approximately 1 , for example, the selection threshold may be calculated 
with p = 1 . The selection threshold setting section 11 6 then outputs (in a step S215) the selection threshold 6 2 . 
[0238] FIG. 46 is a figure showing the processing which is performed by the database feature selection section. 
[0239] Next, referring to FIG. 46, the database feature selection section 117 reads in (in a step S216) the query 
25 histogram which is outputted from the query feature extraction section 101, the classification histograms (clusters) 
which are outputted from the database feature classification section 1 1 5, and the selection threshold which is outputted 
from the selection threshold setting section 116. Next, the database feature selection section 117 calculates the Euclid 
distance between the query histogram which has been read in and the centroid histogram of each cluster on the 
database signal side (in a step S21 7). Next, based upon these distances which it has calculated, the database feature 
30 selection section 1 1 7 selects (in a step S21 8) a cluster which is capable of including a histogram which corresponds 
to the signal to be searched for. The principle by which this is done will be explained below. 

[0240] Here, FIG. 47 is a figure showing the principle by which the histogram space (in the above example, a space 
of 1 28 dimensions) is dissected upon a plane which includes three points Q, C 1 , and C 2 . Here, Q is the query histogram, 
C 1 is the centroid histogram of the cluster to which the histogram Q belongs, and C 2 is the centroid histogram of some 

35 other cluster, while cIq.,, d^, and d 12 are, respectively, the distance between Q and C 1( the distance between Q and 
C 2 , and the distance between C n and C 2 . Here, if it is supposed that it is required to detect the location in the database 
signal which corresponds to the histogram whose distance from the histogram Q is less than d, then a histogram which 
is internal to the hypersphere (shown as a circle in FIG. 47) of radius d centered upon Q corresponds to the location 
in the database signal which it is required to detect. When the radius of the hypersphere centered upon Q becomes 

40 greater than d e , then there is a possibility that the histogram which corresponds to the location in the database signal 
to be detected may be included among the histograms which belong to the cluster which is represented by C 2 . Here, 
when the selection threshold G 2 has become greater than d e , the cluster which is represented by is selected. 
[0241] d e is obtained in the following manner. From FIG. 47, the following Equation holds: 
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From Equation (10): 



55 



2*12 



(11) 



34 



EP 1 426 899 Pag e 35 of 102 



EP 1 426 899 A2 

2 2 

[0242] Accordingly, from Equation (11), when — — — —(12) holds, all the histograms which belong to the 
cluster which is represented by C 2 are selected (in a!?ef> S21 9). These steps are performed for all the clusters except 
for the one to which the histogram Q belongs, and the locations in the database signal which correspond to the histo- 
grams which are selected are outputted (in a step S220). The following procedures are only executed for the locations 

5 which are outputted by the database feature selection section 117. 

[0243] Then, the segment extraction section 1 09 through the database feature compression section 1 11 , the data- 
base feature partitioning section 1 03 through the signal detection decision section 1 08, and the distance re-calculation 
section 113 and the signal detection re-decision section 114 now perform their procedures. Note that, since these 
procedures which are performed by these sections are the same as in the case of the seventh preferred embodiment 

10 which was described above, accordingly their explanation will be curtailed. Note that, according to requirements, the 
distance re-calculation section 113 and the signal detection re-decision section 114 may be included, or, if they are not 
required, may not be included. 

[0244] Next, the query feature compression section 112 reads in the locations in the database signal which have 
been outputted from the database feature selection section 1 1 7, the query histogram which is outputted from the query 

is feature extraction section 1 01 , and the set of linear mappings which are outputted from the compression mapping 
determination section 110. Next, the query feature compression section 112 performs mapping of the histograms to a 
corresponding subspace, using the linear mappings. The mappings are used at the locations which are outputted from 
the database feature selection section 11 7. It is sufficient only to perform this for the linear mappings; it is not necessary 
to do so for all of the linear mappings. By doing this, a noteworthy benefit is obtained with regard to reduction of the 

20 processing time. 

-THE TENTH PREFERRED EMBODIMENT - 

[0245] Next, a tenth preferred embodiment of the present invention will be explained with reference to the drawings. 

25 [0246] FIG. 35 is a functional block diagram showing the structure of a signal retrieval device to which the method 
of this tenth preferred embodiment of the present invention has been applied. With the signal retrieval device of this 
preferred embodiment, in the signal retrieval device according to the ninth preferred embodiment of the present inven- 
tion, there is further comprised a skip width calculation section 118; and it takes as its input a query signal, in other 
words an audio signal which it is desired to retrieve for serving as a pattern or sample, and a database signal, in other 

30 words an audio signal which is retrieved, and the device outputs the location within the database signal for which the 
distance from the query signal is less than a value 6., which been set in advance (this is termed the search threshold). 
[0247] The skip width calculation section 118 calculates the skip width for the window upon which attention is focused 
based upon the distance which is outputted from the distance compensation section 1 07, and shifts the window upon 
which attention is focused by this skip width. Note that the actual details of the operation of the skip width calculation. 

35 section 1 1 8 are the same as in the case of the third preferred embodiment described above, and accordingly they have 
been curtailed herein. 

- RESULTS OF ACTUAL EXPERIMENTS - 

40 [0248] Next, actual experimental results which were obtained during the operation of a signal retrieval device ac- 
cording to this preferred embodiment of the present invention will be explained. 

[0249] Here, the conditions of the experiments were the same as for the seventh preferred embodiment described 
above. As another parameter, the number of clusters was set to 1 024. 

[0250] The results of these experiments are shown in FIG. 50 and FIG. 51. FIG. 50 is a fifth explanatory figure 
45 showing the experimental results. In addition, FIG. 51 is a sixth explanatory figure showing the experimental results. 
The number of segments is, in each case, shown along the horizontal axis; and in FIG. 50 the time period which was 
required for the retrieval is shown along the vertical axis, while in FIG. 51 the number of matches is shown along the 
vertical axis. When the number of segments was taken as 1 0,000, with the method of this preferred embodiment, the 
time period which was required for the retrieval was 0.234 seconds and the number of matches was 305,351 ; while, 
so with a method in which only feature compression was implemented ("using feature compression"), the time period 
which was required for the retrieval was 0.364 seconds and the number of matches was 772,784; and, with a conven- 
tional method (Time-Series Active Search"), the time period which was required for the retrieval was 4.218 seconds 
and the number of matches was 633,047. 

55 - THE ELEVENTH PREFERRED EMBODIMENT - 

[0251 ] Next, an eleventh preferred embodiment of the present invention will be explained with reference to the draw- 
. ings. 
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[0252] FIG. 36 is a functional block diagram of a signal retrieval device to which the method of this eleventh preferred 
embodiment has been applied. With the signal retrieval device of this preferred embodiment, in the signal retrieval 
device according to the fifth preferred embodiment of the present invention, there are further comprised a distance re- 
calculation section 113, a signal detection re-decision section 114, a database feature classification section 115, a 

5 selection threshold setting section 116, and a database feature selection section 1 1 7; and the device takes as its input 
a query signal, in other words an audio signal which it is desired to retrieve for serving as a pattern or sample, and a 
database signal, in other words an audio signal which is retrieved, and the device outputs the location within the da- 
tabase signal for which the distance from the query signal is less than a value e 1 which has been set in advance (this 
is termed the search threshold). 

10 [0253] Note that, with regard to the procedures performed by this preferred embodiment, after the procedure which 
is performed by the query feature extraction section 101 and the procedure which is performed by the database feature 
extraction section 102 have been completed in the manner described above in connection with the fifth preferred 
embodiment, then the steps S210 through S212 of the procedure which is performed by the database feature classi- 
fication section 115, the steps S213 through S215 of the procedure which is performed by the selection threshold 

15 setting section 116, and the steps S216 through S220 of the procedure which is performed by the database feature 
selection section 117 are performed in the manner described above in connection with the ninth preferred embodiment. 
[0254] Next, the steps S1 64 through S1 66 of the procedure which is performed by the database feature partitioning 
section 1 03, the steps S1 67 through S 1 70 of the procedure which is performed by the database feature pruning section 
104, and the steps S 171 through S 173 of the procedure which is performed by the feature region extraction section 

20 1 05 are performed in the manner described above in connection with the fifth preferred embodiment. Next, the feature 
matching section 1 06 reads in the representative feature sequence which is outputted from the database feature prun- 
ing section 104 and the query feature which is outputted from the database feature selection section 117, and the 
procedures shown in the steps S 174 through S 176 are performed. Subsequently, the steps S 177 through S 179 of 
the procedure which is performed by the distance compensation section 107 and the procedure which is performed 

25 by the signal detection decision section 1 08 are performed in the manner described above in connection with the fifth 
preferred embodiment, and the procedures which are performed by the distance re-calculation section 113 and by the 
signal decision re-decision section 114 are performed in the manner described above in connection with the seventh 
preferred embodiment. Note that, according to requirements, the distance re-calculation section 113 and the signal 
detection re-decision section 114 may be included, or, if they are not required, may not be included. 

30 

- THE TWELFTH PREFERRED EMBODIMENT - 

[0255] Next, a twelfth preferred embodiment of the present invention will be described in detail with reference to the 
figures. 

35 [0256] FIG. 37 is a figure showing functional blocks of a signal retrieval device to which a method according to the 
twelfth preferred embodiment has been applied. The signal retrieval device of this preferred embodiment is constructed 
by further adding a skip width calculation section 11 8 to the signal retrieval device of the eleventh preferred embodiment, 
and takes as its input a query signal, in other words an audio signal which has been retrieved and which serves as a 
pattern or sample, and a database signal, in other words an audio signal which is retrieved, and outputs a location in 
the database signal at which the distance from the query signal is less than a value B 1 which has been set in advance 
(this is termed the search threshold value). 

[0257] Here, the skip width calculation section 118 calculates a skip width for the window upon which attention is 
focused which is set for the sequence of DB histograms which is outputted from the database feature extraction section 
102, based upon the distance which has been outputted from the distance compensation section 107, and shifts the 
« window upon which attention is focused by this skip width. The other processing steps which are performed are the 
same as in the case of the eleventh preferred embodiment described above. 

- THE THIRTEENTH PREFERRED EMBODIMENT - 

so [0258] Next, a thirteenth preferred embodiment of the present invention will be explained with reference to the draw- 
ings. 

[0259] Although this preferred embodiment is able to deal with various different types of object signal for processing, 
here, by way of example, an audio signal will be considered as the object signal for processing. 
[0260] FIG. 52 is a functional block diagram showing the structure of a signal retrieval device to which the method 
55 of this thirteenth preferred embodiment has been applied. Referring to this figure, the signal retrieval device of this 
preferred embodiment comprises a query feature extraction section 1 01 , a database feature extraction section 1 02, a 
database feature classification section 115, a selection threshold setting section 116, a database feature selection 
section 117, a segment extraction section 109, a compression mapping determination section 110, a database feature 
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compression section 111, a query feature compression section 112, a feature matching section 106, and a signal 
detection decision section 8. 

[0261] The signal retrieval device shown in FIG. 52 takes as its input a query signal, in other words an audio signal 
which it is desired to retrieve for serving as a pattern or sample, and a database signal, in other words an audio signal 
which is retrieved, and the device outputs the location within the database signal for which the distance from the query 
signal is less than a value 6 1 which been set in advance (this is termed the search threshold). 

[0262] Note that the query feature extraction section 101 produces features from the query signal. The database 
feature extraction section 1 02 sets the window in the database signal upon which attention is focused, and produces 
features while shifting this window upon which attention is focused. Furthermore, the database feature classification 
section 115 classifies the features which have been produced by repeatedly performing the procedure which is per- 
formed by the database feature extraction section 1 02 while shifting the window upon which attention is focused, based 
upon a distance which is defined in advance, and determines upon the representative feature of this classification. The 
selection threshold setting section 1 1 6 calculates a selection threshold for the distance which has been defined by the 
database feature classification section 115 from the search threshold which has been defined in advance. Then the 
database feature selection section 117, for the classification which has been outputted from the database feature 
classification section 115, selects a feature which is included in the classification so that the distance from the feature 
which has been outputted from the query feature extraction section 1 01 satisfies the condition which is produced from 
the selection threshold which has been outputted from the selection threshold setting section 116. 
[0263] Furthermore, the segment extraction section 1 09 extracts a segment, which is a sub-sequence, by segmenting 
the feature sequence which has been produced by repeatedly performing the procedure of the database feature ex- 
traction section 1 02 while shifting the window upon which attention is focused. The compression mapping determination 
section 110 determines, from each segment which has been outputted from the segment extraction section 1 09, upon 
a mapping for calculation of a feature of lower dimensions than the feature. The database feature compression section 
111 calculates a feature of lower dimensions than the feature, which corresponds to the segment which has been 
outputted from the segment extraction section 109, based upon the mapping which has been outputted from the com- 
pression mapping determination section 110. The query feature compression section 112 calculates a feature of lower 
dimensions than the feature, which corresponds to the feature which has been outputted from the query feature ex- 
traction section 1 01 , based upon the mapping which has been outputted from the compression mapping determination 
section 110. 

[0264] Furthermore, the feature matching section 1 06 calculates the distance between the compressed feature se- 
quence which has been outputted from the database feature compression section 111 and the compressed feature 
sequence which has been outputted from the query feature compression section 112, for the location within the data- 
base signal which has been outputted from the database feature selection section 117. The signal detection decision 
section 108 determines whether or not the query signal is present at the location within the database signal, by com- 
paring together the distance which has been outputted from the feature matching section 1 06 and a search threshold, 
which is a threshold which corresponds to the distance. 

[0265] Next, the operation of the signal retrieval device of this preferred embodiment will be explained with reference 
to FIGS. 52 and 55. 

[0266] The procedures which are performed by the query feature extraction section 101 and the database feature 
extraction section 102 are the same as in the case of the second preferred embodiment which was described above. 
In addition, the procedures which are performed by the database feature classification section 1 1 5, the selection thresh- 
old setting section 116, and the database feature selection section 1 1 7 are the same as in the case of the ninth preferred 
embodiment which was described above. Moreover, the procedure which is performed by the segment extraction sec- 
tion 109 is the same as in the case of the sixth preferred embodiment which was described above. Yet further, the 
procedures which are performed by the compression mapping determination section 110, the database feature com- 
pression section 111 , the feature matching section 106, and the signal detection section 108 are the same as in the 
case of the second preferred embodiment which was described above. Finally, the procedure which is performed by 
the query feature compression section 112 is the same as in the case of the ninth preferred embodiment which was . 
described above. 

-THE FOURTEENTH PREFERRED EMBODIMENT - 

[0267] Next, a fourteenth preferred embodiment of the present invention will be explained with reference to the draw- 
ings. 

[0268] FIG. 53 is a functional block diagram showing the structure of a signal retrieval device to which the method 
of this fourteenth preferred embodiment has been applied. With the signal retrieval device of this preferred embodiment, 
in the signal retrieval device according to the thirteenth preferred embodiment, there are further comprised a distance 
re-calculation section 113 and a signal detection re-decision section 114; and the device takes as its input a query 
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signal, in other words an audio signal which it is desired to retrieve for serving as a pattern or sample, and a database 
signal, in other words an audio signal which is retrieved, and the device outputs the location within the database signal 
for which the distance from the query signal is less than a value Q A which been set in advance (this is termed the search 
threshold). 

5 [0269] Here, for the location in the database signal at which it has been decided by the signal detection decision 
section 108 that the query signal is present, the distance re-calculation section 113 calculates the distance between 
the feature sequence which has been outputted from the query feature extraction section 1 01 and the feature sequence 
which has been outputted from the database feature extraction section 1 02. By comparing together the distance which 
has been outputted from the distance re-calculation section 113 and the search threshold, the signal detection re- 

10 decision section 114 re-determines whether or not the query signal is present at the location of the database signal. 
[0270] Note that the procedures which are performed by the distance re-calculation section 113 and by the signal 
detection re-decision section 114 are the same as in the case of the fourth preferred embodiment described above, 
and accordingly their description will be curtailed. 

[0271] Furthermore, since the procedures which are performed by the query feature extraction section 101, the 
is database feature extraction section 1 02, the database feature classification section 1 1 5, the selection threshold setting 
section 1 1 6, the database feature selection section 11 7, the segment extraction section 1 09, the compression mapping 
determination section 1 1 0, the database feature compression section 111, the query feature compression section 112, 
the feature matching section 106, and the signal detection decision section 108 are the same as in the case of the 
thirteenth preferred embodiment described above, therefore their description will be curtailed. 

20 

-THE FIFTEENTH PREFERRED EMBODIMENT - 

[0272] Next, a fifteenth preferred embodiment of the present invention will be explained with reference to the draw- 
ings. 

25 [0273] FIG. 54 is a functional block diagram showing the structure of a signal retrieval device to which the method 
of this fifteenth preferred embodiment has been applied. With the signal retrieval device of this preferred embodiment, 
in the signal retrieval device according to the fourteenth preferred embodiment of the present invention, there is further 
comprised a skip width calculation section 118; and the device takes as its input a query signal, in other words an audio 
signal which it is desired to retrieve for serving as a pattern or sample, and a database signal, in other words an audio 

30 signal which is retrieved, and the device outputs the location within the database signal for which the distance from 
the query signal is less than a value Q A which been set in advance (this is termed the search threshold). 
[0274] The skip width calculation section 1 1 8 calculates the skip width forthe window upon which attention is focused 
based upon the distance which has been outputted from the feature matching section 1 06, and shifts the window upon 
which attention is focused by this skip width. 

35 [0275] Next, the processing which is performed by the signal retrieval device of this preferred embodiment will be 
explained. 

[0276] First, since the processing which is performed by the query feature extraction section 101, the database 
feature extraction section 102, the database feature classification section 115, the selection threshold setting section 
1 1 6, the database feature selection section 117, the segment extraction section 1 09, the compression mapping deter- 

<o mination section 110, the database feature compression section 111 , and the query feature compression section 112, 
and the processing which is performed by the feature matching section 1 06, the signal detection decision section 1 08, 
the distance re-calculation section 113, and the signal detection re-decision section 114 are the same as in the thirteenth 
and the fourteenth preferred embodiments described above, the explanation thereof will be curtailed. Furthermore, the 
processing which is performed by the skip width calculation section 118 is the same as that performed in the third 

45 preferred embodiment described above. 

- RESULTS OF ACTUAL EXPERIMENTS - 

[0277] Next, actual experimental results which were obtained during the operation of a device to which the preferred 

50 embodiment of the present invention was applied will be explained. 

[0278] In order to check upon the beneficial effects of the present invention, first, a histogram which was made from 
about 10 hours of audio signal was taken as a database signal, and investigation was performed of the file size, the 
time period for search, and the number of feature matches when compression features had been written into the file. 
The parameters of the search were: audio signal sampling frequency = 33 kHz; dimensionality of the feature vector = 

55 7; time width of the feature vector = 60 msec; time resolution for the feature vector = 1 0 msec; dimensionality of the 
histogram = 128; width of the time window = 15 sec; contribution threshold = 0.9; number of segments = 200; and 
search threshold = 85. In the feature pruning, sequence partitioning was performed by equi-segmentation, and the 
tops of the sub- histogram sequences were employed as representative features just as they were. The distance meas- 
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ure for the matching was Euclid distance. 

[0279] FIG. 56 is a seventh explanatory figure showing the experimental results. FIG. 57 is a ninth explanatory figure 
showing the experimental results. The experimental results when the width of the sub-histogram sequences was varied 
from 1 0 to 50 by steps of 1 0 are shown in FIG. 56 and FIG. 57. In both cases, the width a of the sub-histogram sequences 
5 is shown along the horizontal axes of the graphs, while in FIG. 56 the file size is shown along the vertical axis, and in 
FIG. 57, the number of matching locations is shown along the vertical axis on the left hand side and the time period 
for the search is shown along the vertical axis on the right hand side. The case a = 0 means that the method of the 
present invention is not being employed. 

[0280] As can be understood from FIG. 56 and FIG. 57, it is possible to monotonically reduce the file size and the 
10 number of matching locations by increasing the width of the sub-histogram sequences. Furthermore, as can be un- 
derstood from FIG. 57, the search time is also reduced along with reduction of the number of matching locations. The 
fact that the search time changes over to increasing at the boundary where a = 40 is because the number of locations 
at which it is necessary to perform re-matching for the histograms increases due to the range over which features are 
present being magnified. . 

is [0281] Due to the above, it is considered that a = 40 is the width of the most suitable sub-histogram sequence, and 
the file size at this time was 5.8 megabytes (about 1/30 of the value when the present invention was not employed), 
the search time was 23 msec (about 60% of the value when the present invention was not employed), and the number 
of matching locations was 41 ,855 (about 85% of the value when the present invention was not employed). 
[0282] As has been explained, according to the present invention, the benefit is obtained that, by performing the 

20 compression in the time domain while pruning the features of the database signal in advance, it is possible to greatly 
reduce the number of indices while ensuring that false dismissal do not occur, so that, in comparison with well-known 
methods, it is possible to perform signal retrieval at higher speed. 

[0283] Note that it would also be possible to perform the signal compression procedure or the signal retrieval pro- 
cedure by recording a program for implementing the function of each of the processing sections in FIGS. 1 through 4 

25 upon a recording medium capable of being read in by a computer system, and by causing this program which has 
been recorded upon the recording medium to be read in by a computer system, thus causing the program to be exe- 
cuted. Note that, by the term "computer system", it is intended to include an operating system and hardware such as 
various peripheral devices. Furthermore, in the case of utilization of the WWW system, the term "computer system" is 
intended also to encompass a presentation environment (or a display environment) for a home page. Yet further, the 

30 term "recording medium capable of being read in by a computer system" is intended to encompass an item of trans- 
portable media such as a flexible disk, an opto-magnetic disk, a ROM, a CD-ROM or the like, or a storage device 
provided within a computer system such as a hard disk or the like. Even further, the term "recording medium capable 
of being read in by a computer system" is also intended to include the case in which the program is retained for a 
certain time period, such as the case in which, when the program is transmitted via a network such as the Internet or. 

35 the like, or via a transmission line such as a telephone line or the like, it is held in a volatile memory (RAM) within a 
computer system which functions as a server or a client. 

[0284] Moreover, the above described program may be transmitted from a computer system upon which this program 
is stored in a storage device or the like to another computer system via a transmitting medium, or by a transmission 
wave within a transmitting medium. Here, the term 'transmission medium" which transmits the program is intended to 
40 mean a medium which is endowed with the function of transmitting information, such as a network (a communication 
network) such as the Internet or the like, or a transmission line such as a telephone line or the like. Furthermore, the 
above described program may be one for implementing only a portion of the above described function. Yet further, it 
would be acceptable to implement the above described function via a differential file (a differential program) which was 
able to do so in combination with a program which was already recorded in a computer system. 

45 

Claims 

1. A signal compression method for compressing an original signal which has been provided in advance to convert 
so the original signal into a compressed signal, comprising: 

an initial sub-signal creation step of creating, from the original signal, sub-signals of shorter length than the 
original signal; 

a created sub-signal selection step of, for each of the sub-signals which have been produced by the initial 
55 sub-signal creation step, pruning the created sub-signal candidates to those for which the amount of data is 

less than for the original signal; 

a sub-signal re-creation step of determining upon a created sub-signal which is actually to be used, using the 
created sub-signal candidates which have been produced by the created sub-signal selection step; 
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a compression mapping determination step of determining, from the respective sub-signals which have been 
produced by the sub-signal re-creation step, a mapping for calculation of a compressed signal; and 
a signal compression step of calculating a compressed signal which corresponds to each of the sub-signals 
which have been obtained by the sub-signal re-creation step, based upon the mapping which has been ob- 
5 tained by the compression mapping determination step. 

2. A signal compression method as described in Claim 1 , wherein the signal compression step comprises: 

a signal mapping step of mapping each of the sub-signals which have been obtained by the sub-signal re- 
10 creation step by the mapping which has been obtained by the compression mapping determi nation step; 

a projection distance calculation step of calculating, for the sub-signal after the mapping which has been 
obtained by the signal mapping step, the distance from the sub-signal which has been obtained by the sub- 
signal re-creation step; and 

a compressed feature creation step of creating a compressed signal, based upon the respective sub-signals 
is after mapping which have been produced by the signal mapping step and the projection distance which has 

been produced by the projection distance calculation step. 

3. A signal compression method as described in Claim 1 , wherein the initial sub-signal creation step segments the 
original signal from the top of the original signal, and takes the sub-signal after the segmentation as its resulting 

20 sub-signal. 

4. A signal compression method as described in Claim 3, wherein the created sub-signal selection step and the sub- 
signal re-creation step determine segmentation boundaries in order from the top of the original signal. 

25 5. a signal compression method as described in Claim 3, wherein the created sub-signal selection step and the sub- 
signal re-creation step set a segmentation boundary shiftable width which is determined in advance, and, taking 
the segmentation boundary which has been obtained by the initial sub-signal creation step as a reference, deter- 
mine segmentation boundaries which are to be actually utilized within a segmentation boundary shiftable range 
having the segmentation boundary shiftable width on both sides of the center thereof. 

30 

6. A signal compression method as described in Claim 3, wherein the created sub-signal selection step shifts the 
segmentation boundaries to some locations and calculates compression ratios, and, based upon the results there- 
of, selects a range in which the segmentation boundaries which are to be actually utilized can exist. 

35 7. A signal compression method as described in Claim 6, wherein the created sub-signal selection step automatically 
determines the number of times for calculation of compression ratio in the created sub-signal selection step, so 
as to reduce the number of times of calculation of compression ratio in the created sub-signal selection step and 
the sub-signal re-creation step. 

40 8. A signal compression method as described in Claim 1 , wherein the initial sub-signal creation step extracts features 
from the original signal, and uses the extracted features, represented as a sequence of multi-dimensional vectors, 
as a new original signal. 

9. A signal retrieval method for, at any location within a stored signal, which is an original signal which is registered 
45 in advance, calculating the distance from a reference signal, which is a signal which is taken as an object, and 

finding a location from the stored signal which is similar to the reference signal, comprising: 

the steps which are comprised in the signal compression method as described in Claim 1 ; 
a reference feature extraction step in which a feature is produced from the reference signal; 
50 a stored feature extraction step in which a window upon which attention is focused is set within the stored 

signal, and in which a feature is produced from the stored signal within the window upon which attention is 
focused; 

a reference feature compression step in which a reference feature which has been produced by the reference 
feature extraction step is compressed, based upon the mapping which has been produced by the compression 
55 mapping determination step; 

a feature matching step in which the distance is calculated between a reference compressed signal which has 
been produced by the reference feature compression step and a stored compressed signal which has been 
produced from the signal compression step by newly using the feature sequence which has been produced 
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by repeatedly performing the processing of the stored feature extraction step while shifting the window upon 
which attention is focused; and 

a signal detection decision step in which, by comparing together the distance which has been produced by 
the feature matching step and a search threshold, which is a threshold which corresponds to the distance, it 
is decided whether or not the reference signal is present at the location within the stored signal, 

wherein the processing of the feature matching step and the processing of the signal detection decision step 
are repeated while shifting the window upon which attention is focused. 

10. A signal retrieval method as described in Claim 9, further comprising: 

a distance re-calculation step in which, for the location in the database signal at which it has been decided by 
the signal detection decision step that the query signal is present, the distance between the feature sequence 
which has been produced by the reference feature extraction step and the feature sequence which has been 
produced by the stored feature extraction step is calculated; and 

a signal detection re-decision step in which, by comparing together the distance which has been produced by 
the distance re-calculation step and the search threshold, it is re-decided whether or not the query signal is 
present at the location of the database signal, 

wherein the processing of the feature matching step, the signal detection decision step, the distance re- 
calculation step, and the signal detection re-decision step is repeated while shifting the window upon which atten- 
tion is focused; for some locations within the database signal, the distance from the query signal is calculated; and 
it is determined whether or not the query signal is present at the locations within the database signal. 

1 1 . A signal retrieval method as described in Claim 9, further comprising a skip width calculation step in which, based 
upon the distance which has been calculated by the feature matching step, a skip width for the window upon which 
attention is focused is calculated, and the window upon which attention is focused is shifted by the skip width, 

wherein the processing of the feature matching step, the signal detection decision step, and the skip width 
calculation step is repeated while shifting the window upon which attention is focused; for some locations within 
the database signal, the distance from the query signal is calculated; and it is determined whether or not the query 
signal is present at the locations within the database signal. 

12. A signal compression device which compresses an original signal which is provided in advance to convert the 
original signal into a compressed signal, comprising: 

an initial sub-signal creation section which creates, from the original signal, sub-signals of shorter length than 
the original signal; 

a created sub-signal selection section which, for each of the sub-signals which has been produced by the 
initial sub-signal creation section, prunes the created sub-signal candidates to those for which the amount of 
data is less than for the original signal; 

a sub-signal re-creation section which, using the created sub-signal candidates which have been produced 
by the created sub-signal selection section, determines upon a created sub-signal which is actually to be used; 
a compression mapping determination section which determines upon a mapping for calculation of a com- 
pressed signal from the respective sub-signals which has been obtained by the sub-signal re-creation section; 
and 

a signal compression section which calculates a compressed signal which corresponds to each of the sub- 
signals which have been obtained by the sub-signal re-creation section, based upon the mapping which has 
been obtained by the compression mapping determination section. 

13. A signal retrieval device for, at any location within a stored signal, which is an original signal which is registered 
in advance, calculating the distance from a reference signal, which is a signal which is taken as an object, and 
finding a location from the stored signal which is similar to the reference signal, comprising: 

the sections which are comprised in the signal compression device as described in Claim 12; 

a reference feature extraction section which produces a feature from the reference signal; 

a stored feature extraction section which sets a window upon which attention is focused within the stored 

signal, and which produces a feature from the stored signal within the window upon which attention is focused; 

a reference feature compression section which compresses a reference feature which has been produced by 
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the reference feature extraction section, based upon the mapping which has been produced by the compres- 
sion mapping determination section; 

a feature matching section which calculates the distance between a reference compressed signal which has 
been produced by the reference feature compression section and a stored compressed signal which has been 
5 produced from the signal compression section by newly using the feature sequence which has been produced 

by repeatedly performing the processing by the stored feature extraction section while shifting the window 
upon which attention is focused; and 

a signal detection decision section which, by comparing together the distance which has been produced by 
the feature matching section and a search threshold, which is a threshold which corresponds to the distance, 
10 decides whether or not the reference signal is present at the location within the stored signal, 

wherein the operation of the feature matching section and the operation of the signal detection decision 
section are repeated while shifting the window upon which attention is focused. 

is 14. a signal retrieval device as described in Claim 13, further comprising: 

a distance re-calculation section which, for the location in the database signal at which it has been decided 
by the signal detection decision section that the query signal is present, calculates the distance between the 
feature sequence which has been produced by the reference feature extraction section and the feature se- 
20 quence which has been produced by the stored feature extraction section; and 

a signal detection re-decision section which, by comparing together the distance which has been produced 
by the distance re-calculation section and the search threshold, re-decides whether or not the query signal is 
present at the location of the database signal, 

25 wherein the processing of the feature matching section, the signal detection decision section, the distance 

re-calculation section, and the signal detection re-decision section is repeated while shifting the window upon 
which attention is focused; for some locations within the database signal, the distance from the query signal is 
calculated; and it is determined whether or not the query signal is present at the locations within the database signal. 

30 15. A signal retrieval device as described in Claim 1 3, further comprising a skip width calculation section which, based 
upon the distance which has been calculated by the feature matching section, calculates a skip width forthe window 
upon which attention is focused, and shifts the window upon which attention is focused by the skip width, and 

wherein the processing of the feature matching section, the signal detection decision section, and the skip 
width calculation section is repeated while shifting the window upon which attention is focused; for some locations 

35 within the database signal, the distance from the query signal is calculated; and it is determined whether or not 

the query signal is present at the locations within the database signal. 

16. A signal compression program which compresses an original signal which has been provided in advance to convert 
the original signal into a compressed signal, for causing a computer to execute: 

40 

an initial sub-signal creation step of creating, from the original signal, sub-signals of shorter length than the 
original signal; 

a created sub-signal selection step of, for each of the sub-signals which have been produced by the initial 
sub-signal creation step, pruning the created sub-signal candidates to those for which the amount of data is 

4 5 less than for the original signal; 

a sub-signal re-creation step of determining upon a created sub-signal which is actually to be used, using the 
created sub-signal candidates which have been produced by the created sub-signal selection step; 
a compression mapping determination step of determining, from each of the sub-signals which have been 
produced by the sub-signal re-creation step, a mapping for calculation of a compressed signal; and 

50 a signal compression step of calculating a compressed signal which corresponds to each of the sub-signals 

which have been obtained by the sub-signal re-creation step, based upon the mapping which has been ob- 
tained by the compression mapping determination step. 

17. A signal retrieval program for, at any location within a stored signal, which is an original signal which is registered 
55 in advance, calculating the distance from a reference signal, which is a signal which is taken as an object, and 

finding a location from the stored signal which is similar to the reference signal, for causing a computer to execute: 

the steps which are comprised in the signal compression program as described in Claim 1 6; 
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a reference feature extraction step in which a feature is produced from the reference signal; 

a stored feature extraction step in which a window upon which attention is focused is set within the stored 

signal, and in which a feature is produced from the stored signal within the window upon which attention is 

focused; 

a reference feature compression step in which a reference feature which has been produced by the reference 
feature extraction step is compressed, based upon the mapping which has been produced by the compression 
mapping determination step; 

a feature matching step in which the distance is calculated between a reference compressed signal which has 
been produced by the reference feature compression step and a stored compressed signal which has been 
produced from the signal compression step by newly using the feature sequence which has been produced 
by repeatedly performing the processing of the stored feature extraction step while shifting the window upon 
which attention is focused; 

a signal detection decision step in which, by comparing together the distance which has been produced by 
the feature matching step and a search threshold, which is a threshold which corresponds to the distance, it 
is decided whether or not the reference signal is present at the location within the stored signal; and 
a step of repeatedly executing the feature matching step and the signal detection decision step while shifting 
the window upon which attention is focused. 

A signal retrieval program as described in Claim 1 7, further comprising: 

a distance re-calculation step in which, for the location in the database signal at which it has been decided by 
the signal detection decision step that the query signal is present, the distance between the feature sequence 
which has been produced by the reference feature extraction step and the feature sequence which has been 
produced by the stored feature extraction step is calculated; and 

a signal detection re-decision step in which, by comparing together the distance which has been produced by 
the distance re-calculation step and the search threshold, it is re-decided whether or not the query signal is 
present at the location of the database signal, and 

wherein a step is performed in which the processing of the feature matching step, the signal detection decision 
step, the distance re-calculation step, and the signal detection re-decision step is repeated while shifting the window 
upon which attention is focused, for some locations within the database signal, the distance from the query signal 
is calculated; and it is determined whether or not the query signal is present at the locations within the database 
signal. 

A signal retrieval program as described in Claim 17, further comprising a skip width calculation step in which, 
based upon the distance which has been calculated by the feature matching step, a skip width for the window 
upon which attention is focused is calculated, and the window upon which attention is focused is shifted by the 
skip width, and 

wherein a step is performed in which the processing of the feature matching step, the signal detection decision 
step, and the skip width calculation step is repeated while shifting the window upon which attention is focused, for 
some locations within the database signal, the distance from the query signal is calculated; and it is determined 
whether or not the query signal is present at the locations within the database signal. 

A recording medium capable of being read by a computer, upon which is recorded a signal compression program 
which compresses an original signal which has been provided in advance to convert the original signal into a 
compressed signal, for causing a computer to execute: 

an initial sub-signal creation step of creating, from the original signal, sub-signals of shorter length than the 
original signal; 

a created sub-signal selection step of, for each of the sub-signals which have been produced by the initial 
sub-signal creation step, pruning the created sub-signal candidates to those for which the amount of data is 
less than for the original signal; 

a sub-signal re-creation step of determining upon a created sub-signal which is actually to be used, using the 

created sub-signal candidates which have been produced by the created sub-signal selection step; 

a compression mapping determination step of determining, from each of the sub-signals which have been 

produced by the sub-signal re-creation step, a mapping for calculation of a compressed signal; and 

a signal compression step of calculating a compressed signal which corresponds to each of the sub-signals 

which have been obtained by the sub-signal re-creation step, based upon the mapping which has been ob- 
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tained by the compression mapping determination step. 

21. A recording medium capable of being read by a computer, upon which is recorded a signal retrieval program for, 
at any location within a stored signal, which is an original signal which is registered in advance, calculating the 

5 distance from a reference signal, which is a signal which is taken as an object, and finding a location from the 

stored signal which is similar to the reference signal, for causing a computer to execute: 

the steps which are comprised in the signal compression program as described in Claim 20; 
a reference feature extraction step in which a feature is produced from the reference signal; 
10 a stored feature extraction step in which a window upon which attention is focused is set within the stored 

signal, and in which a feature is produced from the stored signal within the window upon which attention is 
focused; 

a reference feature compression step in which a reference feature which has been produced by the reference 
feature extraction step is compressed, based upon a mapping which has been produced by the compression 

15 mapping determination step; 

a feature matching step in which the distance is calculated between a reference compressed signal which has 
been produced by the reference feature compression step and a stored compressed signal which has been 
produced from the signal compression step by newfy using the feature sequence which has been produced 
by repeatedly performing the processing of the stored feature extraction step while shifting the window upon 

20 which attention is focused; 

a signal detection decision step in which, by comparing together the distance which has been produced by 
the feature matching step and a search threshold, which is a threshold which corresponds to the distance, it 
is decided whether or not the reference signal is present at the location within the stored signal; and 
a step of repeatedly executing the feature matching step and the signal detection decision step while shifting 

25 the window upon which attention is focused. 

22. A recording medium capable of being read by a computer upon which is recorded the signal retrieval program as 
described in Claim 21 , the signal retrieval program further comprising: 

30 a distance re-calculation step in which, for the location in the database signal at which it has been decided by 

the signal detection decision step that the query signal is present, the distance between the feature sequence 
which has been produced by the reference feature extraction step and the feature sequence which has been 
produced by the stored feature extraction step is calculated; and 

a signal detection re-decision step in which, by comparing together the distance which has been produced by. 
35 the distance re-calculation step and the search threshold, it is re-decided whether or not the query signal is 

present at the location of the database signal, and 

wherein a step is performed in which the processing of the feature matching step, the signal detection decision 
step, the distance re-calculation step, and the signal detection re-decision step is repeated while shifting the window 
40 upon which attention is focused, for some locations within the database signal, the distance from the query signal 

is calculated; and it is determined whether or not the query signal is present at the locations within the database 
signal. 

23. A recording medium capable of being read by a computer upon which is recorded the signal retrieval program as 
45 described in Claim 21 , the signal retrieval program further comprising a skip width calculation step in which, based 

upon the distance which has been calculated by the feature matching step, a skip width for the window upon which 
attention is focused is calculated, and the window upon which attention is focused is shifted by the skip width, and 
wherein a step is performed in which the processing of the feature matching step, the signal detection decision 
step, and the skip width calculation step is repeated while shifting the window upon which attention is focused, for 
50 some locations within the database signal, the distance from the query signal is calculated, and it is determined 

whether or not the query signal is present at the locations within the database signal. 

24. A signal retrieval method which finds out portions from a database signal which has been registered in advance 
which are similar to a query signal which is taken as an object, comprising: 

55 

a query feature extraction step in which a feature is produced from the query signal; 

a database feature extraction step in which a window upon which attention is focused is set within the database 
signal, and in which a feature is produced from the database signal within the window upon which attention 
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is focused; 

a database feature partitioning step in which a feature sequence which has been produced by repeatedly 
performing the database feature extraction step while shifting the window upon which attention is focused is 
partitioned; 

a database feature pruning step in which a representative feature is extracted from the feature sequence which 
has been obtained after partitioning by the database feature partitioning step, and a representative feature 
sequence is produced which consists of a smaller number of features; 

afeature region extraction step in which a region is produced in which a feature which is included in the partition 
which has been produced by the database feature partitioning step is present; 

afeature matching step in which a distance is calculated between a feature sequence which has been produced 
by the query feature extraction step and a representative feature sequence which has been produced by the 
databas e f eatu re pru n i n g step ; 

a distance compensation step in which the distance which has been calculated by the feature matching step 
is compensated using the region which has been produced by the feature region extraction step; and 
a signal detection decision step in which, by comparing together the distance which has been produced after 
compensation by the distance compensation step and a search threshold, which is a threshold which corre- 
sponds to the distance, it is decided whether or not the q uery signal is present at the location within the database 
signal, and 

wherein the processing of the feature matching step through the signal detection decision step is repeated 
while shifting the window upon which attention is focused, for some locations within the database signal, the dis- 
tance from the query signal is calculated, and it is determined whether or not the query signal is present at the 
locations within the database signal. 

25. A signal retrieval method as described in Claim 24, wherein, in the database feature pruning step, any single 
feature in the partition is taken as a representative feature. 

26. A signal retrieval method as described in Claim 24, wherein, in the database feature pruning step, the centroid of 
the features in the partition is taken as a representative feature. 

27. A signal retrieval method as described in Claim 24, wherein, in the database feature partitioning step, the feature 
sequence which has been produced by repeatedly performing the processing of the database feature extraction 
step while shifting the window upon which attention is focused is segmented equally into lengths which have been 
specified in advance. 

28. A signal retrieval method as described in Claim 24, wherein, in the database feature partitioning step, the feature 
sequence which has been produced by repeatedly performing the processing of the database feature extraction 
step while shifting the window upon which attention is focused is segmented so that the region in which a feature 
is present which is produced by the feature region extraction step becomes smaller than a maximum region which 
has been specified in advance. 

29. A signal retrieval method as described in Claim 24, further comprising: 

a segment extraction step in which segments, which are sub-sequences, are extracted by segmenting a feature 
sequence which has been produced by repeatedly performing the database feature extraction step while shift- 
ing the window upon which attention is focused; 

a compression mapping determination step in which, from each of the segments which have been obtained 
by the segment extraction step, a mapping is determined for calculation of a feature of less dimensions than 
the feature; 

a database feature compression step in which a feature which corresponds to a segment which has been 
obtained by the segment extraction step and which is of less dimensions than the feature is calculated based 
upon the mapping which has been obtained by the compression mapping determination step; and 
a query feature compression step in which a feature which corresponds to a feature which has been obtained 
by the query feature extraction step and which is of less dimensions than the feature is calculated based upon 
the mapping which has been obtained by the compression mapping determination step, and 

wherein, in the database feature pruning step, a representative feature sequence is produced by taking the 
compressed feature sequence which has been produced by the database feature compression step as a new 
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feature sequence, and, in the feature matching step, a matching is performed of the compressed feature which 
has been produced by the query feature compression step as a new feature, the processing of the feature matching 
step through the signal detection decision step is repeated while shifting the window upon which attention is fo- 
cused, for some locations within the database signal, the distance from the query signal is calculated; and it is 
5 determined whether or not the query signal is present at the locations within the database signal. 

30. A signal retrieval method as described in Claim 29, wherein the database feature compression step comprises: 

a database feature mapping step in which a segment which has been obtained by the segment extraction step 
is mapped according to the mapping which has been obtained by the compression mapping determination step; 
a database projection distance calculation step in which, for the compressed feature sequence which has 
been produced by the database feature mapping step, the distance from the feature sequence which has been 
produced by the database feature extraction step is calculated; and 

a database compressed feature creation step in which a new compressed feature sequence is created from 
the compressed feature sequence which has been produced by the database feature mapping step and the 
projection distance which has been produced by the database projection distance calculation step, and wherein 
the query feature compression step comprises: 

a query feature mapping step in which the feature which has been obtained by the query feature extraction 
step is mapped according to the mapping which has been obtained by the compression mapping deter- 
mination step; 

a query projection distance calculation step in which, for the compressed feature which has been produced 
by the query feature mapping step, the distance from the feature which has been produced by the query 
feature extraction step is calculated; and 

a query compression feature creation step in which a new compressed feature is created from the com- 
pressed feature which has been produced by the query feature mapping step and the projection distance 
which has been produced by the query projection distance calculation step. 

31. A signal retrieval method as described in Claim 29, wherein the compression mapping determination step extracts 
30 a representative feature by a Karhunen-Loeve transform. 

32. A signal retrieval method as described in Claim 24, further comprising: 

a distance re-calculation step in which, for the location in the database signal at which it has been decided by 
the signal detection decision step that the query signal is present, the distance between the feature which has 
been produced by the query feature extraction step and the feature sequence which has been produced by 
the database feature extraction step is calculated; and 

a signal detection re-decision step in which, by comparing together the distance which has been produced by 
the distance re-calculation step and the search threshold, it is again decided whether or not the query signal 
is present at the location of the database signal, and 

wherein the processing of the feature matching step, the signal compensation step, the signal detection 
decision step, the distance re-calculation step, and the signal detection re-decision step is repeated while shifting 
the window upon which attention is focused, for some locations within the database signal, the distance from the 
query signal is calculated; and it is determined whether or not the query signal is present at the locations within 
the database signal. 

33. A signal retrieval method as described in Claim 24, further comprising: 

a database feature classification step in which the respective features which have been produced by repeatedly 
performing the database feature extraction step while shifting the window upon which attention is focused are 
classified based upon a distance which has been defined in advance, and a representative feature of the 
classification is determined upon; 

a selection threshold setting step in which a selection threshold for the distance which has been defined by 
the database feature classification step is calculated from a search threshold which has been defined in ad- 
vance; and 

a database feature selection step in which, among the classification which has been produced by the database 
feature classification step, a feature is selected which is included in the classification which contains a repre- 
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sentative feature such that the distance from the feature which has been produced by the query feature ex- 
traction step satisfies a condition which is produced from the selection threshold which has been calculated 
by the selection threshold setting step. 

5 34. A signal retrieval method as described in Claim 29, further comprising: 

a database feature classification step in which the respective features which have been produced by repeatedly 
performing the database feature extraction step while shifting the window upon which attention is focused are 
classified based upon a distance which has been defined in advance, and a representative feature of the 
10 classification is determined upon; 

a selection threshold setting step in which a selection threshold for the distance which has been defined by 
the database feature classification step is calculated from a search threshold which has been defined in ad- 
vance; and 

a database feature selection step in which, among the classification which has been produced by the database 
15 feature classification step, a feature is selected which is included in the classification which contains a repre- 

sentative feature such that the distance from the feature which has been produced by the query feature ex- 
traction step satisfies a condition which is produced from the selection threshold which has been calculated 
by the selection threshold setting step. 

20 35. a signal retrieval method as described in Claim 33, wherein the database feature classification step classifies the 
features based upon a vector quantization algorithm, using Euclid distance as a distance measure. 

36. A signal retrieval method as described in Claim 24, wherein the feature matching step calculates the distance 
based upon Manhattan distance or Euclid distance. 

25 

37. A signal retrieval method as described in Claim 30, wherein the database projection distance calculation step 
calculates the distance based upon Manhattan distance or Euclid distance. 

38. A signal retrieval method as described in Claim 32, wherein the distance re-calculation step calculates the distance 
30 based upon Manhattan distance or Euclid distance. 

39. A signal retrieval method as described in Claim 24, wherein the query feature extraction step and the database 
feature extraction step classify the features by a method which is determined in advance, create a histogram which 
is a frequency distribution table for each classification, and output the histogram as a new feature. 

35 

40. A signal retrieval method as described in Claim 24, further comprising a skip width calculation step in which, based 
upon the distance which has been calculated by the distance compensation step, a skip width for the window upon 
which attention is focused is calculated, and the window upon which attention is focused is shifted by the skip 
width, and 

40 wherein the processing of the feature matching step, the distance compensation step, the signal detection 

decision step, and the skip width calculation step is repeated while shifting the window upon which attention is 
focused, for some locations within the database signal, the distance from the query signal is calculated, and it is 
determined whether or not the query signal is present at the locations within the database signal. 

4* 41. A signal retrieval method which finds out portions from a database signal which has been registered in advance 
which are similar to a query signal which is taken as an object, comprising: 

a query feature extraction step in which a feature is produced from a query signal; 

a database feature extraction step in which a window upon which attention is focused is set within the database 
50 signal, and in which a feature is produced from the database signal within the window upon which attention 

is focused; 

a database feature classification step in which the respective features which have been produced by repeatedly 
performing the database feature extraction step while shifting the window upon which attention is focused are 
classified based upon a distance which has been defined in advance, and a representative feature of the 
55 classification is determined upon; 

a selection threshold setting step in which a selection threshold for the distance which has been defined by 
the database feature classification step is calculated from a search threshold which has been defined in ad- 
vance; 



47 



EP 1 426 899 Page 48 of 103 



EP 1 426 899 A2 

a database feature selection step in which, among the classification which has been produced by the database 
feature classification step, a feature is selected which is included in the classification which contains a repre- 
sentative feature such that the distance from the feature which has been produced by the query feature ex- 
traction step satisfies a condition which is produced from the selection threshold which has been calculated 
5 by the selection threshold setting step; 

a segment extraction step in which segments, which are sub-sequences, are extracted by segmenting a feature 
sequence which has been produced by repeatedly performing the database feature extraction step while shift- 
ing the window upon which attention is focused; 

a compression mapping determination step in which, from each of the segments which have been obtained 
10 by the segment extraction step, a mapping is determined for calculation of a feature of less dimensions than 

the feature; 

a database feature compression step in which a feature which corresponds to a segment which has been 
obtained by the segment extraction step and which is of less dimensions than the feature is calculated based 
upon the mapping which has been obtained by the compression mapping determination step; 

15 a query feature compression step in which a feature which corresponds to a feature which has been obtained 

by the query feature extraction step and which is of less dimensions than the feature is calculated based upon 
the mapping which has been obtained by the compression mapping determination step; 
a feature matching step in which a distance is calculated between a compressed feature sequence which has 
been produced by the database feature compression step and a compressed feature which has been produced 

20 by the query feature extraction step; and 

a signal detection decision step in which, by comparing together the distance which has been calculated by 
the feature matching step and a search threshold, which is a threshold which corresponds to the distance, it 
is decided whether or not the query signal is present at the location within the database signal, and 

25 wherein the processing of the feature matching step and the signal detection decision step is repeated while 

shifting the window upon which attention is focused, for some locations within the database signal, the distance 
from the query signal is calculated; and it is determined whether or not the query signal is present at the locations 
within the database signal. 

30 42. A signal retrieval method as described in Claim 41 , further comprising: 

a distance re-calculation step in which, for the location in the database signal at which it has been decided by 
the signal detection decision step that the query signal is present, the distance between the feature which has 
been produced by the query feature extraction step and the feature sequence which has been produced by 
35 the database feature extraction step is calculated; and 

a signal detection re-decision step in which, by comparing togetherthe distance which has been produced by 
the distance re-calculation step and the search threshold, it is again decided whether or not the query signal 
is present at the location of the database signal, and 

40 wherein the processing of the feature matching step, the signal detection decision step, the distance re- 

calculation step, and the signal detection re-decision step is repeated while shifting the window upon which atten- 
tion is focused, for some locations within the database signal, the distance from the query signal is calculated; and 
it is determined whether or not the query signal is present at the locations within the database signal. 

45 43. a signal retrieval method as described in Claim 41 , further comprising a skip width calculation step in which, based 
upon the distance which has been calculated by the feature matching step, a skip width for the window upon which 
attention is focused is calculated, and the window upon which attention is focused is shifted by the skip width, and 
wherein the processing of the feature matching step, the signal detection decision step, and the skip width 
calculation step is repeated while shifting the window upon which attention is focused, for some locations within 

50 the database signal, the distance from the query signal is calculated; and it is determined whether or not the query 

signal is present at the locations within the database signal. 

44. A signal retrieval device which finds out portions from a database signal which has been registered in advance 
which are similar to a query signal which is taken as an object, comprising: 
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a query feature extraction section which produces a feature from the query signal; 

a database feature extraction section which sets a window upon which attention is focused is set within the 
database signal, and which produces a feature from the database signal within the window upon which atten- 
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tion is focused; 

a database feature partitioning section which partitions a feature sequence which has been produced by re- 
peatedly performing the processing of the database feature extraction section while shifting the window upon 
which attention is focused; 

a database feature pruning section which extracts a representative feature from the feature sequence which 
has been obtained after partitioning by the database feature partitioning section, and which produces a rep- 
resentative feature sequence which consists of a smaller number of features; 

a feature region extraction section which produces a region in which a feature which is included in the partition 
which has been produced by the database feature partitioning section is present; 

a feature matching section which calculates a distance between a feature sequence which has been produced 
by the query feature extraction section and a representative feature sequence which has been produced by 
the database feature pruning section; 

a distance compensation section in which the distance which has been calculated by the feature matching 
section is compensated using the region which has been produced by the feature region extraction section; and 
a signal detection decision section in which, by comparing together the distance which has been produced 
after compensation by the distance compensation section and a search threshold, which is a threshold which 
corresponds to the distance, it is decided whether or not the query signal is present at the location within the 
database signal, and 

wherein the processing of the feature matching section through the signal detection decision section is re- 
peated while shifting the window upon which attention is focused; for some locations within the database signal, 
the distance from the query signal is calculated; and it is determined whether or not the query signal is present at 
the locations within the database signal. 

45. A signal retrieval device as described in Claim 44, further comprising: 

a segment extraction section which extracts segments, which are sub-sequences, by segmenting a feature 
sequence which has been produced by repeatedly performing the processing of the database feature extrac- 
tion section while shifting the window upon which attention is focused; 

a compression mapping determination section which, from each of the segments which have been obtained 
by the segment extraction section, determines a mapping for calculation of a feature of less dimensions than 
the feature; 

a database feature compression section which calculates a feature which corresponds to a segment which 
has been obtained by the segment extraction section and which is of less dimensions than the feature based, 
upon a mapping which has been obtained by the compression mapping determination section; and 
a query feature compression section which calculates a feature which corresponds to a feature which has 
been obtained by the query feature extraction section and which is of less dimensions than the feature based 
upon a mapping which has been obtained by the compression mapping determination section, and 

wherein, by the database feature pruning section, a representative feature sequence is produced by using 
a compressed feature sequence which has been produced by the database feature compression section is pro- 
duced as a new feature sequence; by the feature matching section, matching is performed using a compressed 
feature which has been produced by the query feature compression section as a new feature; and further: the 
processing of the feature matching section through the signal detection decision section is repeated while shifting 
the window upon which attention is focused, for some locations within the database signal, the distance from the 
query signal is calculated; and it is determined whether or not the query signal is present at the locations within 
the database signal. 

46. A signal retrieval device as described in Claim 44, further comprising: 

a distance recalculation section which, for the location in the database signal at which it has been decided 
by the signal detection decision section that the query signal is present, calculates the distance between the 
feature sequence which has been produced by the query feature extraction section and the feature sequence 
which has been produced by the database feature extraction section; and 

a signal detection re-decision section which, by comparing together the distance which has been produced 
by the distance re-calculation section and the search threshold, again decides whether or not the query signal 
is present at the location of the database signal, and 
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wherein the processing of the feature matching section, the distance compensation section, the signal de- 
tection decision section, the distance re-calculation section, and the signal detection re-decision section is repeated 
while shifting the window upon which attention is focused, for some locations within the database signal, the dis- 
tance from the query signal is calculated; and it is determined whether or not the query signal is present at the 
5 locations within the database signal. 

47. A signal retrieval device as described in Claim 44, further comprising: 

a database feature classification section which classifies the respective features which have been produced 
10 by repeatedly performing the processing of the database feature extraction section while shifting the window 

upon which attention is focused based upon a distance which has been determined in advance, and determines 
upon a representative feature of the classification; 

a selection threshold setting section which calculates a selection threshold for the distance which has been 
defined by the database feature classification section from a search threshold which has been defined in 

15 advance; and 

a database feature selection section which, for the classification which has been produced by the database 
feature classification section, selects a feature which is included in the classification which contains a repre- 
sentative feature such that the distance from the feature which has been produced by the query feature ex- 
traction section satisfies a condition which is produced from the selection threshold which has been calculated 

20 by the selection threshold setting section. 

48. A signal retrieval device as described in Claim 44, further comprising a skip width calculation section which, based 
upon the distance which has been calculated by the distance compensation section, calculates a skip width for 
the window upon which attention is focused, and shifts the window upon which attention is focused by the skip 

25 width, and 

wherein the processing of the feature matching section, the distance compensation section, the signal de- 
tection decision section, and the skip width calculation section is repeated while shifting the window upon which 
attention is focused, for some locations within the database signal, the distance from the query signal is calculated; 
and it is determined whether or not the query signal is present at the locations within the database signal. 

30 

49. A signal retrieval device which finds out portions from a database signal which has been registered in advance 
which are similar to a query signal which is taken as an object, comprising: 

a query feature extraction section which produces a feature from a query signal; 
35 a database feature extraction section which sets a window upon which attention is focused within the database 

signal, and which produces a feature from the database signal within the window upon which attention is 
focused; 

a database feature classification section which classifies the respective features which have been produced 
by repeatedly performing the processing of the database feature extraction section while shifting the window 
40 upon which attention is focused based upon a distance which has been determined in advance, and determines 

upon a representative feature of the classification; 

a selection threshold setting section which calculates a selection threshold for the distance which has been 
defined by the database feature classification section from a search threshold which has been defined in 
advance; 

45 a database feature selection section which, among the classification which has been produced by the database 

feature classification section, selects a feature which is included in the classification which contains a repre- 
sentative feature such that the distance from the feature which has been produced by the query feature ex- 
traction section satisfies a condition which is produced from the selection threshold which has been calculated . 
by the selection threshold setting section; 

50 a segment extraction section which extracts segments, which are sub-sequences, by segmenting a feature 

sequence which has been produced by repeatedly performing the processing of the database feature extrac- 
tion section while shifting the window upon which attention is focused; 

a compression mapping determination section which, from each of the segments which have been obtained 
by the segment extraction section, determines a mapping for calculation of a feature of less dimensions than 
55 the feature; 

a database feature compression section which calculates a feature which corresponds to a segment which 
has been obtained by the segment extraction section and which is of less dimensions than the feature based 
upon a mapping which has been obtained by the compression mapping determination section; 
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a query feature compression section which calculates a feature which corresponds to a feature which has 
been obtained by the query feature extraction section and which is of less dimensions than the feature based 
upon the mapping which has been obtained by the compression mapping determination section; 
a feature matching section which calculates a distance between a compressed feature sequence which has 
been produced by the database feature compression section and a compressed feature which has been pro- 
duced by the query feature extraction section; and 

a signal detection decision section which, by comparing together the distance which has been calculated by 
the feature matching section and a search threshold, which is a threshold which corresponds to the distance, 
decides whether or not the query signal is present at the location within the database signal, and 

wherein the processing of the feature matching section through the signal detection decision section is re- 
peated while shifting the window upon which attention is focused, for some locations within the database signal, 
the distance from the query signal is calculated; and it is determined whether or not the query signal is present at 
the locations within the database signal. 

50. A signal retrieval device as described in Claim 49, further comprising: 

a distance re-calculation section which, for the location in the database signal at which it has been decided 
by the signal detection decision section that the query signal is present, calculates the distance between the 
feature sequence which has been produced by the query feature extraction section and the feature sequence 
which has been produced by the database feature extraction section; and 

a signal detection re-decision section which, by comparing together the distance which has been produced 
by the distance re-calculation section and the search threshold, again decides whether or not the query signal 
is present at the location of the database signal and 

wherein the processing of the feature matching section, the signal detection decision section, the distance 
re-calculation section, and the signal detection re-decision section is repeated while shifting the window upon 
which attention is focused, for some locations within the database signal, the distance from the query signal is 
calculated; and it is determined whether or not the query signal is present at the locations within the database signal. 

51 . A signal retrieval device as described in Claim 49, further comprising a skip width calculation section which, based 
upon the distance which has been calculated by the feature matching section, calculates a skip width forthe window 
upon which attention is focused, and shifts the window upon which attention is focused by the skip width and 

wherein the processing of the feature matching section, the signal detection decision section, and the skip 
width calculation section is repeated while shifting the window upon which attention is focused; for some locations 
within the database signal, the distance from the query signal is calculated; and it is determined whether or not 
the query signal is present at the locations within the database signal. 

52. A program for execution by a computer of a signal retrieval device which finds out portions from a database signal 
which has been registered in advance which are similar to a query signal which is taken as an object, comprising: 

a query feature extraction step in which a feature is produced from the query signal; 

a database feature extraction step in which a window upon which attention is focused is set within the database 
signal, and in which a feature is produced from the database signal within the window upon which attention 
is focused; 

a database feature partitioning step in which a feature sequence which has been produced by repeatedly 
performing the database feature extraction step while shifting the window upon which attention is focused is 
partitioned; 

a database feature pruning step in which a representative feature is extracted from the feature sequence which 
has been obtained after partitioning by the database feature partitioning step, and a representative feature 
sequence is produced which consists of a smaller number of features; 

afeature region extraction step in which a region is produced in which afeature which is included in the partition 
which has been produced by the database feature partitioning step is present; 

a feature matching step in which a distance is calculated between a feature which has been produced by the 
query feature extraction step and a representative feature sequence which has been produced by the database 
feature pruning step; 

a distance compensation step in which the distance which has been calculated by the feature matching step 
is compensated using the region which has been produced by the feature region extraction step; 
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a signal detection decision step in which, by comparing together the distance which has been produced after 
compensation by the distance compensation step and a search threshold, which is a threshold which corre- 
sponds to the distance, it is decided whether or not the query signal is present at the location within the database 
signal; and 

a step of: repeating the processing of the feature matching step through the signal detection decision step 
while shifting the window upon which attention is focused, for some locations within the database signal, cal- 
culating the distance from the query signal; and determining whether or not the query signal is present at the 
locations within the database signal. 

53. A program as described in Claim 52, further comprising: 

a segment extraction step in which segments, which are sub-sequences, are extracted by segmenting a feature 
sequence which has been produced by repeatedly performing the database feature extraction step while shift- 
ing the window upon which attention is focused; 

a compression mapping determination step in which, from each of the segments which have been obtained 
by the segment extraction step, a mapping is determined for calculation of a feature of less dimensions than 
the feature; 

a database feature compression step in which a feature which corresponds to a segment which has been 
obtained by the segment extraction step and which is of less dimensions than the feature is calculated based 
upon the mapping which has been obtained by the compression mapping determination step; 
a query feature compression step in which a feature which corresponds to a feature which has been obtained 
by the query feature extraction step and which is of less dimensions than the feature is calculated based upon 
the mapping which has been obtained by the compression mapping determination step; and 
a step in which, in the database feature pruning step, a representative feature sequence is produced by taking 
the compressed feature sequence which has been produced by the database feature compression step as a 
new feature sequence, and, in the feature matching step, a matching is performed of the compressed feature 
which has been produced by the query feature compression step as a new feature; the processing of the 
feature matching step through the signal detection decision step is repeated while shifting the window upon 
which attention is focused; for some locations within the database signal, the distance from the query signal 
is calculated; and it is determined whether or not the query signal is present at the locations within the database 
signal. 

54. A program as described in Claim 52, further comprising: 

a distance re-calculation step in which, for the location in the database signal at which it has been decided by 
the signal detection decision step that the query signal is present, the distance between the feature which has 
been produced by the query feature extraction step and the feature sequence which has been produced by 
the database feature extraction step is calculated; 

a signal detection re-decision step in which, by comparing together the distance which has been produced by 
the distance re-calculation step and the search threshold, it is again decided whether or not the query signal 
is present at the location of the database signal; and 

a step in which: the processing of the feature matching step, the distance compensation step, the signal de- 
tection decision step, the distance re-calculation step, and the signal detection re-decision step is repeated 
while shifting the window upon which attention is focused; for some locations within the database signal, the 
distance from the query signal is calculated; and it is determined whether or not the query signal is present at 
the locations within the database signal. 

55. A program as described in Claim 52, further comprising: 

a database feature classification step in which the respective features which have been produced by repeatedly 
performing the database feature extraction step while shifting the window upon which attention is focused are 
classified based upon a distance which has been determined in advance, and a representative feature of the 
classification is determined upon; 

a selection threshold setting step in which a selection threshold for the distance which has been defined by 
the database feature classification step is calculated from a search threshold which has been defined in ad- 
vance; and 

a database feature selection step in which, among the classification which has been produced by the database 
feature classification step, a feature is selected which is included in the classification which contains a repre- 
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sentatlve feature such that the distance from the feature which has been produced by the query feature ex- 
traction step satisfies a condition which is produced from the selection threshold which has been calculated 
by the selection threshold setting step. 

56. A program as described in Claim 52, further comprising: 

a skip width calculation step in which, based upon the distance which has been calculated by the distance 
compensation step, a skip width for the window upon which attention is focused is calculated, and the window 
upon which attention is focused is shifted by the skip width; and 

a step in which: the processing of the feature matching step, the distance compensation step, the signal de- 
tection decision step, and the skip width calculation step is repeated while shifting the window upon which 
attention is focused; for some locations within the database signal, the distance from the query signal is cal- 
culated; and it is determined whether or not the query signal is present at the locations within the database 
signal. 

57. A program for execution by a computer of a signal retrieval device which finds out portions from a database signal 
which has been registered in advance which are similar to a query signal which is taken as an object, comprising: 

a query feature extraction step in which a feature is produced from the query signal; 

a database feature extraction step in which a window upon which attention is focused is set within the database 
signal, and in which a feature is produced from the database signal within the window upon which attention 
is focused; 

a database feature classification step in which the respective features which have been produced by repeatedly 
performing the database feature extraction step while shifting the window upon which attention is focused are 
classified based upon a distance which has been determined in advance, and a representative feature of the 
classification is determined upon; 

a selection threshold setting step in which a selection threshold for the distance which has been defined by 
the database feature classification step is calculated from a search threshold which has been defined in ad- 
vance; 

a database feature selection step in which, among the classification which has been produced by the database 
feature classification step, a feature is selected which is included in the classification which contains a repre- 
sentative feature such that the distance from the feature which has been produced by the query feature ex- 
traction step satisfies a condition which is produced from the selection threshold which has been calculated 
by the selection threshold setting step; 

a segment extraction step in which segments, which are sub-sequences, are extracted by segmenting a feature 
sequence which has been produced by repeatedly performing the database feature extraction step while shift- 
ing the window upon which attention is focused; 

a compression mapping determination step in which, from each of the segments which have been obtained 
by the segment extraction step, a mapping is determined for calculation of a feature of less dimensions than 
the feature; 

a database feature compression step in which a feature which corresponds to a segment which has been 

obtained by the segment extraction step and which is of less dimensions than the feature is calculated based 

upon the mapping which has been obtained by the compression mapping determination step; 

a query feature compression step in which a feature which corresponds to a feature which has been obtained 

by the query feature extraction step and which is of less dimensions than the feature is calculated based upon 

the mapping which has been obtained by the compression mapping determination step; 

a feature matching step in which a distance is calculated between a compressed feature sequence which has 

been produced by the database feature compression step and a compressed feature which has been produced . 

by the query feature extraction step; 

a signal detection decision step in which, by comparing together the distance which has been calculated by 
the feature matching step and a search threshold, which is a threshold which corresponds to the distance, it 
is decided whether or not the query signal is present at the location within the database signal; and 
a step in which: the processing of the feature matching step and the signal detection decision step is repeated 
while shifting the window upon which attention is focused; for some locations within the database signal, the 
distance from the query signal is calculated; and it is determined whether or not the query signal is present at 
the locations within the database signal. 

58. A program as described in Claim 57, further comprising: 
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a distance re-calculation step in which, for the location in the database signal at which it has been decided by 
the signal detection decision step that the query signal is present, the distance between the feature which has 
been produced by the query feature extraction step and the feature sequence which has been produced by 
the database feature extraction step is calculated; 
5 a signal detection re-decision step in which, by comparing together the distance which has been produced by 

the distance re-calculation step and the search threshold, it is again decided whether or not the query signal 
is present at the location of the database signal; and 

a step in which: the processing of the feature matching step, the signal detection decision step, the distance 
re-calculation step, and the signal detection re-decision step is repeated while shifting the window upon which 
10 attention is focused; for some locations within the database signal, the distance from the query signal is cal- 

culated; and it is determined whether or not the query signal is present at the locations within the database 
signal. 

59. A program as described in Claim 57, further comprising: 

15 

a skip width calculation step in which, based upon the distance which has been calculated by the feature 
matching step, a skip width for the window upon which attention is focused is calculated, and the window upon 
which attention is focused is shifted by the skip width; and 

a step in which: the processing of the feature matching step, the signal detection decision step, and the skip 
20 width calculation step is repeated while shifting the window upon which attention is focused; for some locations 

within the database signal, the distance from the query signal is calculated; and it is determined whether or 
not the query signal is present at the locations within the database signal. 

60. A recording medium capable of being read by a computer, upon which is recorded a program for causing a computer 
25 of a signal retrieval device which finds out portions from a database signal which has been registered in advance 

which are similar to a query signal which is taken as an object to execute: 

a query feature extraction step in which a feature is produced from the query signal; 

a database feature extraction step in which a window upon which attention is focused is set within the database 
30 signal, and in which a feature is produced from the database signal within the window upon which attention 

is focused; 

a database feature partitioning step in which a feature sequence which has been produced by repeatedly 
performing the database feature extraction step while shifting the window upon which attention is focused is 
partitioned; 

35 a database feature pruning step in which a representative feature is extracted from the feature sequence which 

has been obtained after partitioning by the database feature partitioning step, and a representative feature 
sequence is produced which consists of a smaller number of features; 

a feature region extraction step in which a region is produced in which a feature which is included in the partition 
which has been produced by the database feature partitioning step is present; 
40 a feature matching step in which a distance is calculated between a feature which has been produced by the 

query feature extraction step and a representative feature sequence which has been produced by the database 
feature pruning step; 

a distance compensation step in which the distance which has been calculated by the feature matching step 
is compensated using the region which has been produced by the feature region extraction step; 
45 a signal detection decision step in which, by comparing together the distance which has been produced after 

compensation by the distance compensation step and a search threshold, which is a threshold which corre- 
sponds to the distance, it is decided whether or not the query signal is present at the location within the database 
signal; and 

a step of: repeating the processing of the feature matching step through the signal detection decision step 
50 while shifting the window upon which attention is focused; for some locations within the database signal, cal- 

culating the distance from the query signal; and determining whether or not the query signal is present at the 
locations within the database signal. 

61 . A recording medium capable of being read by a computer as described in Claim 60, further comprising: 

55 

a segment extraction step in which segments, which are sub-sequences, are extracted by segmenting a feature 
sequence which has been produced by repeatedly performing the database feature extraction step while shift- 
ing the window upon which attention is focused; 



54 



BP 1 426 899 



Page 55 of 103 



EP 1 426 899 A2 

a compression mapping determination step in which, from each of the segments which haves been obtained 
by the segment extraction step, a mapping is determined for calculation of a feature of less dimensions than 
the feature; 

a database feature compression step in which a feature which corresponds to a segment which has been 
5 obtained by the segment extraction step and which is of less dimensions than the feature is calculated based 

upon the mapping which has been obtained by the compression mapping determination step; 
a query feature compression step in which a feature which corresponds to a feature which has been obtained 
by the query feature extraction step and which is of less dimensions than the feature is calculated based upon 
the mapping which has been obtained by the compression mapping determination step; and 
10 a step in which, in the database feature pruning step, a representative feature sequence is produced by taking 

the compressed feature sequence which has been produced by the database feature compression step as a 
new feature sequence, and, in the feature matching step, a matching is performed of the compressed feature 
which has been produced by the query feature compression step as a new feature; the processing of the 
feature matching step through the signal detection decision step is repeated while shifting the window upon 
is which attention is focused; for some locations within the database signal, the distance from the query signal 

is calculated; and it is determined whether or not the query signal is present at the locations within the database 
signal. 

62. A recording medium capable of being read by a computer as described in Claim 60, further comprising: 

20 

a distance re-calculation step in which, for the location in the database signal at which it has been decided by 
the signal detection decision step that the query signal is present, the distance between the feature which has 
been produced by the query feature extraction step and the feature sequence which has been produced by 
the database feature extraction step is calculated; 
25 a signal detection re-decision step in which, by comparing together the distance which has been produced by 

the distance re-calculation step and the search threshold, it is again decided whether or not the query signal 
is present at the location of the database signal; and 

a step in which: the processing of the feature matching step, the distance compensation step, the signal de- 
tection decision step, the distance re-calculation step, and the signal detection re-decision step is repeated 
30 while shifting the window upon which attention is focused; for some locations within the database signal, the 

distance from the query signal is calculated; and it is determined whether or not the query signal is present at 
the locations within the database signal. 

63. A recording medium capable of being read by a computer as described in Claim 60, further comprising: 

35 

a database feature classification step in which the respective features which have been produced by repeatedly 
performing the database feature extraction step while shifting the window upon which attention is focused are 
classified based upon a distance which has been determined in advance, and a representative feature of the 
classification is determined upon; 
40 a selection threshold setting step in which a selection threshold for the distance which has been defined by 

the database feature classification step is calculated from a search threshold which has been defined in ad- 
vance; and 

a database feature selection step in which, among the classification which has been produced by the database 
feature classification step, a feature is selected which is included in the classification which contains a repre- 
ss sentative feature such that the distance from the feature which has been produced by the query feature ex- 
traction step satisfies a condition which is produced from the selection threshold which has been calculated 
by the selection threshold setting step. 

64. A recording medium capable of being read by a computer as described in Claim 60, further comprising: 

50 

a skip width calculation step in which, based upon the distance which has been calculated by the distance 
compensation step, a skip width for the window upon which attention is focused is calculated, and the window 
upon which attention is focused is shifted by the skip width; and 

a step in which: the processing of the feature matching step, the distance compensation step, the signal de- 
tection decision step, and the skip width calculation step is repeated while shifting the window upon which 
attention is focused; for some locations within the database signal, the distance from the query signal is cal- 
culated; and it is determined whether or not the query signal is present at the locations within the database 
signal. 
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65. A recording medium capable of being read by a computer, upon which is recorded a program for causing a computer 
of a signal retrieval device which finds out portions from a database signal which has been registered in advance 
which are similar to a query signal which is taken as an object to execute: 

5 a query feature extraction step in which a feature is produced from the query signal; 

a database feature extraction step in which a window upon which attention is focused is set within the database 
signal, and in which a feature is produced from the database signal within the window upon which attention 
is focused; 

a database feature classification step in which the respective features which have been produced by repeatedly 
10 performing the database feature extraction step while shifting the window upon which attention is focused are 

classified based upon a distance which has been determined in advance, and a representative feature of the 
classification is determined upon; 

a selection threshold setting step in which a selection threshold for the distance which has been defined by 
the database feature classification step is calculated from a search threshold which has been defined in ad- 
's vance; 

a database feature selection step in which, among the classification which has been produced by the database 
feature classification step, a feature is selected which is included in the classification which contains a repre- 
sentative feature such that the distance from the feature which has been produced by the query feature ex- 
traction step satisfies a condition which is produced from the selection threshold which has been calculated 
20 by the selection threshold setting step; 

a segment extraction step in which segments, which are sub-sequences, are extracted by segmenting a feature 
sequence which has been produced by repeatedly performing the database feature extraction step while shift- 
ing the window upon which attention is focused; 

a compression mapping determination step in which, from each of the segments which have been obtained 
25 by the segment extraction step, a mapping is determined for calculation of a feature of less dimensions than 

the feature; 

a database feature compression step in which a feature which corresponds to the segment which has been 
obtained by the segment extraction step and which is of less dimensions than the feature is calculated based 
upon the mapping which has been obtained by the compression mapping determination step; 

30 a query feature compression step in which a feature which corresponds to a feature which has been obtained 

by the query feature extraction step and which is of less dimensions than the feature is calculated based upon 
the mapping which has been obtained by the compression mapping determination step; 
a feature matching step in which a distance is calculated between a compressed feature sequence which has 
been produced by the database feature compression step and a compressed feature which has been produced 

35 by the query feature extraction step; 

a signal detection decision step in which, by comparing together the distance which has been calculated by 
the feature matching step and a search threshold, which is a threshold which corresponds to the distance, it 
is decided whether or not the query signal is present at the location within the database signal; and 
a step in which: the processing of the feature matching step and the signal detection decision step is repeated 

40 while shifting the window upon which attention is focused; for some locations within the database signal, the 

distance from the query signal is calculated; and it is determined whether or not the query signal is present at 
the locations within the database signal. 

66. A recording medium capable of being read by a computer as described in Claim 65, further comprising: 

45 

a distance re-calculation step in which, for the location in the database signal at which it has been decided by 
the signal detection decision step that the query signal is present, the distance between the feature which has 
been produced by the query feature extraction step and the feature sequence which has been produced by 
the database feature extraction step is calculated; 
50 a signal detection re-decision step in which, by comparing together the distance which has been produced by 

the distance re-calculation step and the search threshold, it is again decided whether or not the query signal 
is present at the location of the database signal; and 

a step in which: the processing of the feature matching step, the signal detection decision step, the distance 
re-calculation step, and the signal detection re-decision step is repeated while shifting the window upon which 
55 attention is focused; for some locations within the database signal, the distance from the query signal is cal- 

culated; and it is determined whether or not the query signal is present at the locations within the database 
signal. 
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67. A recording medium capable of being read by a computer as described in Claim 65, further comprising: 

a skip width calculation step in which, based upon the distance which has been calculated by the feature 
matching step, a skip width for the window upon which attention is focused is calculated, and the window upon 
5 which attention is focused is shifted by the skip width; and 

a step in which: the processing of the feature matching step, the signal detection decision step, and the skip 
width calculation step is repeated while shifting the window upon which attention is focused; for some locations 
within the database signal, the distance from the query signal is calculated; and it is determined whether or 
not the query signal is present at the locations within the database signal. 
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